{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"16ff358d-5599-5fdc-9b4e-43d24373f845","task_key":"default--validation--27147","task_revision_id":"1","upstream_id":"27147","short_description":"HellaSwag validation 27147","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to maintain confidentiality [title] Handle private documents carefully. [step] Don't leave private documents unattended, whether you're a nurse, lawyer, or an employee handling your company's confidential material. Remain aware of where the materials are and who can access them.\",\"endings\":\"[\\\"[substeps] For example, if you're a healthcare professional, don't leave a patient's medical records unattended on a cafeteria table or in a reception area. [title] Lock file rooms, cabinets, and other storage spaces.\\\", \\\"This will make you a better candidate to protect your company's confidential material from person-to-person communications or requests for documentation. [substeps] If you are a nurse's aide, be aware of the president's policy on confidentiality.\\\", 'Also make sure your employees are safe, as these are confidential documents that can be accessed on your company\\\\'s public computer system. Reading and recording confidential documents can signal to your employees, and you\\\\'ll need to learn to \\\" train \\\" your employees in the proper manner.', 'Put them on a regular document binder for all employees. [substeps] Remember, there is no law prohibiting disclosure of confidential material.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}