{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"b598d6e7-58fa-5bb8-9285-09a533c73e85","task_key":"default--validation--1117","task_revision_id":"2","upstream_id":"1117","short_description":"HellaSwag validation 1117","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"A group of people are standing in the middle of a forest with a trail of pebbles separating them from a beautiful crystal clear body of water. the camera\",\"endings\":\"['then goes right side up and shows a lot of scenery on the left side.', 'follows the couple kayaking behind the group on the trail moving back and fourth on the murky water in the forest.', 'moves across the water and several people on boats and jet-skis are moving in the water near a large wave.', 'zooms in on the foamy waters as well as the nearby cliffs and forest surrounding it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}