{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7aad35f8-4fb7-5406-ac38-6defbb07dbde","task_key":"default--validation--12684","task_revision_id":"1","upstream_id":"12684","short_description":"HellaSwag validation 12684","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to reduce accidents in the workplace [title] Put formal safety policies and procedures in place. [step] Create a company handbook that lists out the steps that must take place in order to prevent accidents in the work place. Include instructions such as how to store dangerous and toxic items and where certain product should be stowed to ensure safe storage and retrieval.\",\"endings\":\"[\\\"[substeps] Many companies will email you detailed safety policies so that you understand what they are and that you'll always have them. If you are concerned about a potential spill, call the company to report the accident.\\\", 'Also, be sure to let employees know the rules for the workplace. [substeps] Some locations may not have properly trained staff so be sure to inform management beforehand.', '[title] Create strategies for how to maintain employee morale. [step] Develop a policy for how to keep yourself from getting in trouble.', '[title] Put someone in charge of safety in your company. [step] Discuss the current safety policies with this safety coordinator, and work on a plan to make sure that they are adhered to.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}