{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75854074-1040-5ab5-88cf-5c5c3d8ff034","task_key":"default--validation--38301","task_revision_id":"1","upstream_id":"38301","short_description":"HellaSwag validation 38301","config":"default","split":"validation","body":"{\"activity_label\":\"Playing lacrosse\",\"ctx\":\"The team runs to the left and then to the right. A man hits another man with the lacrosse stick and knocks the man down. a man\",\"endings\":\"['runs all the way down the field to the other side and another man throws the ball in the net and makes a goal.', 'shakes his hand profusely and he climbs back up and puts his gaze on roof and talks in front of a wall or a window more quickly.', 'with a towel runs to the right and brings out another player shuffling and winding back to the right.', 'walks up to the batter rip him in half and then stop the men on the field.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}