{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36a34675-c924-50a9-8b09-1972e4b6c07b","task_key":"default--validation--13999","task_revision_id":"1","upstream_id":"13999","short_description":"HellaSwag validation 13999","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man is seen bending over a large set of weights with another walking behind him. The man lifts the bar up over his head and throws it back down. the man\",\"endings\":\"['continues lifting the weights over his head walking backwards.', 'continues shaking the bar and ends by holding a pose in the end.', 'kicks it with his knees while the camera captures him from several angles as the body weight hangs over his head and he throws it down several more times more.', 'does this again while people watch on the sides.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}