{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49e41eff-0648-50ca-b980-18b8b1b5f11b","task_key":"default--validation--47294","task_revision_id":"1","upstream_id":"47294","short_description":"HellaSwag validation 47294","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"The teens are aligned in two lines, then they pass the ball each other to threw the ball to the basket. the coach\",\"endings\":\"['then instructs the teens to shoulder up and play defense while the coach sits on the floor.', 'stands with the teens and discuss before they each round up.', 'is standing on their right as they pass the ball, then the kids return to theirs.', 'gives instructions to the players, then the teens continue practicing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}