{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a23abe3a-8235-5f03-a879-edbcb34b7104","task_key":"default--validation--10673","task_revision_id":"1","upstream_id":"10673","short_description":"HellaSwag validation 10673","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"Several clips are shown of people lifting heavy weights as well as jumping on a track. people\",\"endings\":\"['cheer as they perform quick jumps and their score is shown at the end.', 'are seen jumping over poles, squatting with weights, as well as performing various other exercises.', 'run and do jumps while also landing on the mat.', 'walk up and around the track as well as climbing up a sheer rock face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}