{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38d4411f-f82c-591f-9d43-4e921d48b211","task_key":"default--validation--4579","task_revision_id":"1","upstream_id":"4579","short_description":"HellaSwag validation 4579","config":"default","split":"validation","body":"{\"activity_label\":\"Shuffleboard\",\"ctx\":\"A young boy has a long stick in had and is pushing a puck across a white triangle with numbers in it. The puck goes to far down the board and the toddler chases after it. once the puck\",\"endings\":\"['is won, the toddlers are chased by the mickey mouse dog and they handle all of the puck and they switch places, still pushing it across the black triangle.', 'is back on the board, he takes off running back down the board.', \\\"is in and he can't get it, he moves it back on front him to the opponent.\\\", 'goes near the end, the boy jumps off and picks it back up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}