{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06b13235-d3a5-5132-a524-6dcfb34cd5d6","task_key":"default--validation--24906","task_revision_id":"1","upstream_id":"24906","short_description":"HellaSwag validation 24906","config":"default","split":"validation","body":"{\"activity_label\":\"Snatch\",\"ctx\":\"In the second round, the athlete with blue uniform rise successfully and kiss the bar of the weight, also the other athletes are successful. On the third round, the indian athlete let fall the weight. the athlete with red uniform\",\"endings\":\"['rise score, but he falls at the end.', 'is congratulated, then stand on the first place of the podium and talks.', 'raised and was released the weight bar, he and his other teammates then she jumped to the ground and watches.', 'and white uniform raised but, no weight, the blue uniform fell and also dropped the weight, then off the weight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}