{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"322efdba-6ae5-51ec-9daf-2ae94e4c645b","task_key":"default--validation--44919","task_revision_id":"1","upstream_id":"44919","short_description":"HellaSwag validation 44919","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to determine who is at fault in a car accident [title] Note any traffic citations or violations of traffic law. [step] Any driver violating traffic law will be held largely responsible for a resulting car accident. If one of the drivers is issued a citation for speeding, running a light, or another violation, he or she will likely be at fault.\",\"endings\":\"['[substeps] A citation for \\\" attempted to \\\" in front of someone is \\\" alleged \\\" and can result in the police officer denies the violation of traffic law. You can try to report a violation of the law yourself, but don\\\\'t.', 'If no police are present to issue citations, judge for yourself whether an obvious infraction has occurred. Consult local traffic laws, as these may be different from state laws.', \\\"[substeps] Follow the traffic laws at your local law library, or better yet, stop into your town, city, or county recorder's office. Use the search engine at http: // www.citoyalls.com.\\\", '[title] Note the time and date the vehicle was towed. [step] The orange indicating \\\" towed \\\" will indicate that the vehicle was towed because it isn\\\\'t used for any reason.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}