{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e613a107-4744-573e-9a90-a7bed2e0dcc6","task_key":"default--validation--34494","task_revision_id":"1","upstream_id":"34494","short_description":"HellaSwag validation 34494","config":"default","split":"validation","body":"{\"activity_label\":\"Sumo\",\"ctx\":\"The men then bow before one another as people continue to sweep the arena and lead into the men fighting. the men\",\"endings\":\"['gather at the outside of the arena as people continue to dance and laugh.', 'continuously wrestle one another while the ref moves around them and ends with them walking around then off stage.', 'fight over the other as well as people applauding and the entire crowd clapping.', 'continue to fight with several people watching on the side but ends with one dying.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}