{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5c31766-0343-58a0-9931-eb41525536b5","task_key":"default--validation--10453","task_revision_id":"1","upstream_id":"10453","short_description":"HellaSwag validation 10453","config":"default","split":"validation","body":"{\"activity_label\":\"Table soccer\",\"ctx\":\"A few people are seen standing around various fooseball tables playing games with one another. the people\",\"endings\":\"['hold out bowls and pans in to one another and point to one another.', 'stick hands into cups before pulling them together.', 'continue to push the ball around back and fourth while the camera focuses on one game.', 'continue hitting the ball back and fourth to one another while one throws the ball up in the air and runs away.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}