{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1388cd1b-1f81-575c-b62f-6c4f3d7c320f","task_key":"default--validation--10551","task_revision_id":"1","upstream_id":"10551","short_description":"HellaSwag validation 10551","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to report a kidnapping [title] Contact local law enforcement. [step] If you know or suspect that someone has been kidnapped, contact your local law enforcement agency immediately. The police or sheriff's department will take the information you provide and investigate the situation.\",\"endings\":\"['[substeps] If possible, get a police report by phone or in person at the police station. The actual contact information for the person you suspect has been kidnapped will vary from state to state.', 'They can also contact the fbi or other agencies if necessary. To make a report, call 911 or contact the agency directly.', \\\"Once you've provided sufficient information, a local police officer may begin to investigate the situation. It usually takes a few minutes for a local officer to do this.\\\", '[substeps] Bring the date and time of the abduction to the attention of local law enforcement agencies. Bring any information you have about the person with the stolen vehicle as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}