{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f92554d1-b4b9-5d57-8aa3-4348cb0c68c3","task_key":"default--validation--3079","task_revision_id":"1","upstream_id":"3079","short_description":"HellaSwag validation 3079","config":"default","split":"validation","body":"{\"activity_label\":\"Doing motocross\",\"ctx\":\"While he his talking, several videos of him are shown off road motorcycle racing around the course jumping over humps. however\",\"endings\":\"[', before he finishes his performance, another video holding about 100% of them show him jumping over deep humps inside a building and talking to the camera.', ', most of them are shown without people watching them.', \\\", he isn't shown racing only wearing a helmet as he records himself passing very high lights.\\\", ', he comes back during the segments to show how he uses techniques in certain jumps and the correct way to do it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}