{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ccfe3083-b82d-5d64-80f8-b0025c2e30b0","task_key":"default--validation--31706","task_revision_id":"1","upstream_id":"31706","short_description":"HellaSwag validation 31706","config":"default","split":"validation","body":"{\"activity_label\":\"Kayaking\",\"ctx\":\"A woman is seen speaking to the camera that leads into clips of people riding in kayaks and camels wandering around. More clips are shown of people riding around the water in a kayak. the person\",\"endings\":\"['continues riding along the water while many fall off in the end and the camera pans around the camera.', 'continues speaking to the camera while more clips are shown of people riding themselves or leaving their raft to ride along the water.', 'continues riding all around the area as well as capturing their movements and laughing to the camera.', 'paddles themselves along the water with other people riding near him.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}