{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11a5453d-2db2-5d1f-a800-26c1de5069a5","task_key":"default--validation--33262","task_revision_id":"1","upstream_id":"33262","short_description":"HellaSwag validation 33262","config":"default","split":"validation","body":"{\"activity_label\":\"Beer pong\",\"ctx\":\"One of the men throws a beer pong ball across the table, into a cup. The other man with him throws a ball across the table, too. two other men\",\"endings\":\"['stand on the far right side of the beer pong table, and one of the men reaches to grab the cup where the ball fell into, and the other man bends down to catch a ball that has fallen on the floor.', 'serve and a game of table foosball continues.', 'play table tennis while a man in a red shirt watches their green cups.', 'throw a ball across the table at the same time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}