{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"522e747c-0737-52e8-936c-dccd700b021e","task_key":"default--validation--1301","task_revision_id":"2","upstream_id":"1301","short_description":"HellaSwag validation 1301","config":"default","split":"validation","body":"{\"activity_label\":\"Using parallel bars\",\"ctx\":\"The male athlete is standing by the two poles, raised his arms up, then jumped on the poles with his arms, while under him, a man pull out a red beam. the athlete\",\"endings\":\"['crosses parallel, then stand on all fours above, and on his arms then jump.', 'placed his feet on the pole and then hold it, then jumped from the pole on the poles stop and put his hands on his knees, bend down, and begin swinging on the pole.', 'perform his exhibition on the poles, he stand on his arms while his feet and legs are extended straight up in the air, he turns around, swirl, jumped and twist around then jumped on the mat, standing straight.', 'looked behind him, then and continue walking to the pole.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}