{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b2ab879-0d92-50a2-a503-423eadfc0689","task_key":"default--validation--41770","task_revision_id":"1","upstream_id":"41770","short_description":"HellaSwag validation 41770","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A brunette woman newscaster is wearing a white shirt and royal blue blazer over it and is speaking into the camera. a clip\",\"endings\":\"['attached to the lacrosse game shows the various dips caught in the game and the spectators join in.', 'shows a man who holds a surfboard in his hands and begins playing with the body of the board and begins to tamper with the head area of the board as well as the water and way the scenery is shown.', 'begins and it features people running and the words on the screen say they are novices who are training to run a marathon in tel aviv, israel.', 'of a man tightening a shop front shopping bags and exits spills over the counter and people bend over to check it out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}