{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"350f9f50-ffe3-560b-8218-622c13c277c8","task_key":"default--validation--12949","task_revision_id":"1","upstream_id":"12949","short_description":"HellaSwag validation 12949","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to report vandalism [title] Call your local emergency services to report active vandalism. [step] Do this whether the vandalism is being committed against public or private property. If you catch someone in the act of spray painting a building, stealing a street sign, or otherwise destroying or vandalizing property, call your local emergency services number immediately.\",\"endings\":\"['[title] Report the vandalism by reporting the vandalism. [step] This method will require no explanation of the vandalism.', \\\"The us department of buildings (ftc) has a map at https: // www.ftc.gov/free-contractor/disputes /. [substeps] If you see an officer or official police report vandalism, don't get upset or panic.\\\", 'Destruction of property is illegal, and the authorities will want to catch the person committing the crime if at all possible. [substeps] In the united states you should call 9-1-1 to report vandalism in progress.', '[substeps] A 911 call is a fast, easy way to report vandalism. If you have been on the scene for a year or more, it may be worth calling your local report center.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}