{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c275fd21-e195-5849-b3f2-d3f8ee57f207","task_key":"default--validation--4011","task_revision_id":"1","upstream_id":"4011","short_description":"HellaSwag validation 4011","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"Then, three more boys run to practice the same steps. A young man wearing a yellow t-shirt practice basketball on the other basketball goal. after\",\"endings\":\"[', the getting coach tell them to keep practicing, while trying to make basketball goals without creating obstacles.', ', the young man plays the soccer ball, and the boys walk with the soccer ball.', ', two men wearing playing clothes practice practicing basketball as spectators.', ', a group of three boys runs to practice basketball shooting and blocking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}