{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9110096a-c3f7-599b-ad52-4464e57a2d8f","task_key":"default--validation--4010","task_revision_id":"1","upstream_id":"4010","short_description":"HellaSwag validation 4010","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"Three teens run and a boy throws the basketball to the basket, then the second teen grabs the ball and pass the ball to two teens disputing to have the ball. Then, three more boys run to practice the same steps. a young man\",\"endings\":\"['wearing a yellow t-shirt practice basketball on the other basketball goal.', 'wearing black santa claus suit give the ball to one of the young teens who take it, while his mother pushes the round ball to the other direction.', 'practice on a gym floor, while three other boys practice in gym gym.', 'play basketball with a kid while facing backwards.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}