{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"040235c8-cd90-5c18-974b-2b681646ada8","task_key":"default--validation--25917","task_revision_id":"1","upstream_id":"25917","short_description":"HellaSwag validation 25917","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"The woman n white is instructing the girls in blue uniform on how to do a lay up. The girl one by one rent to the basketball and threw the balls. the girls\",\"endings\":\"['dribble the ball as then do a lay up to the basket.', 'in blue skateboarded and girls in white are walked up stairs around the gym.', 'did the same back and fourth to the man on the monitor.', 'continue to do back flips and doing shots, and the little girl jumps and splits on the grass.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}