{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3021e384-823b-5a61-add1-71150b59ab51","task_key":"default--validation--5384","task_revision_id":"1","upstream_id":"5384","short_description":"HellaSwag validation 5384","config":"default","split":"validation","body":"{\"activity_label\":\"Beach soccer\",\"ctx\":\"A small group of people are seen shaking hands and standing together followed by a man blowing a whistle and people running around. the people\",\"endings\":\"['then play a game of soccer with one another, running up and down the sandy field while coaches yell on the side and a goal is blocked.', 'continue to run around under the stage and close up of a man speaking and shows people playing a game.', 'move around with drums and hitting a tune in the middle of the gym and the man continues to blow a whistle and lead into more people walking past on the side.', 'continue riding around on skateboards as well as following behind one another as well as people speaking to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}