{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56c0a804-97c7-5d67-b76c-0d8132e0ef52","task_key":"default--validation--27354","task_revision_id":"1","upstream_id":"27354","short_description":"HellaSwag validation 27354","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"First the boy pumps something onto his hand from a bottle and he rubs it on his face. then he\",\"endings\":\"[\\\"places items into a bucket, adds soap and scrubs it on the boy's face.\\\", \\\"separates his helmet's parts and uses it to adjust some switches and that's how he is afterwards.\\\", 'washes it off using the water from the bathroom faucet and he looks in the mirror to know how much of it is gone.', 'sprays the areas on his face with a cleaner and then rubs it on the kids face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}