{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c349862-c578-58be-933b-488b57778b71","task_key":"default--validation--33619","task_revision_id":"1","upstream_id":"33619","short_description":"HellaSwag validation 33619","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"There is ia group of runners starting a marathon taking their time. It was scenic and beautiful out there, flowers blossoming and everything. a few of the runners\",\"endings\":\"['are from the hispanic maybach family.', 'get just a bit nervous as they are competing for the gold but eventually they win the race.', 'talk about their experience to reporters.', 'go over the mountains at random as some of them feel the cold off of their trails.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}