{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"112e83b9-82a0-53e1-bf64-a3b3a230439c","task_key":"default--validation--30002","task_revision_id":"1","upstream_id":"30002","short_description":"HellaSwag validation 30002","config":"default","split":"validation","body":"{\"activity_label\":\"Beach soccer\",\"ctx\":\"A large group of people are seen playing a soccer match on a sandy field with many watching on the side. the camera\",\"endings\":\"['continues to follow the group and shows many throwing the ball to one another and the game being played for spectators.', 'captures the game from all angles and watches as one scores a goal into the net.', \\\"follows the opposing teams and leads into them kicking and pulling one another's arms.\\\", 'shows close ups of players as well as rackets and hitting the ball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}