{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26d8ab00-2450-5daa-a97f-2ba2c3b1eb57","task_key":"default--validation--13122","task_revision_id":"1","upstream_id":"13122","short_description":"HellaSwag validation 13122","config":"default","split":"validation","body":"{\"activity_label\":\"Elliptical trainer\",\"ctx\":\"A camera pans away from a set of exercise equipment and leads into a woman's feet moving along. the woman\",\"endings\":\"['progresses along her journey and ends by turning around and continuing on the floor.', 'continues to work on the first step by bending forward, and moving her feet.', 'is then seen using the machine as well as adjusting the settings and continuing to run.', 'removes her sneakers and socks and runs her foot along the bench.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}