{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1435945-3a67-5444-84df-fe065305d741","task_key":"default--validation--28434","task_revision_id":"1","upstream_id":"28434","short_description":"HellaSwag validation 28434","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"Several clips are shown of people playing basketball with one another followed by two men performing drills with one another. the men\",\"endings\":\"['continue to kick the ball around and aim the game in a coordinated manner.', 'show clips again of the man holding three balls and hitting a goal.', 'continue demonstrating how to perform proper basketball shoots while blocking the other.', 'continue to hold the ball back and fourth while gesturing to one another with their hands and standing up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}