{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41f8fec4-bffb-5c18-9786-7c65f1540dc8","task_key":"default--validation--29199","task_revision_id":"1","upstream_id":"29199","short_description":"HellaSwag validation 29199","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"The camera follows a woman as she joins a queue. The woman the camera is following acquires ice cream from a truck. several individuals\",\"endings\":\"['are shown at the store.', 'are shown eating ice cream and talking to the camera.', \\\"stand arms in an embrace as the woman's ice cream container is being taken apart.\\\", 'receive ice cream and rub it in their faces as they sit in a truck.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}