{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"29c214bd-6483-5524-bf5d-488c3ebb4d85","task_key":"default--validation--398","task_revision_id":"2","upstream_id":"398","short_description":"HellaSwag validation 398","config":"default","split":"validation","body":"{\"activity_label\":\"Gargling mouthwash\",\"ctx\":\"A young boy and girl are standing over a sink with their mother talking. the mother\",\"endings\":\"[\\\"is now putting lipstick on the boy's mouth and he tries to stop her but only gets a little bit stuck.\\\", 'washes the boy while the child grabs shampoo.', 'shows some shaving cream on the boy who is scrubbing her brushes.', 'instructs them on how to brush their teeth while laughing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}