{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56d047f0-05da-586d-999f-42c196bc593a","task_key":"default--validation--4351","task_revision_id":"1","upstream_id":"4351","short_description":"HellaSwag validation 4351","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"At the 2008 olympics, a man prepares to lift a loaded barbell. he\",\"endings\":\"['bends down and works his way up with great effort, finally lifting the barbell over his head.', 'lifts it up to his chest and rests it against his shoulder.', 'lifts it over his head and his hands.', 'lifts it up to his chest and looks to see how high it goes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}