{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4ee2b15-07fe-54a0-8a2d-8d5e1660e9a9","task_key":"default--validation--18903","task_revision_id":"1","upstream_id":"18903","short_description":"HellaSwag validation 18903","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"After, the woman pours the black liquid on glasses that have a strainer on top, in one glass the liquid is black and in the other glass the water is clean. after\",\"endings\":\"[', the woman adds the mixture in her bowl, after adds the a powder and the liquid in her solution on the glasses then pour it on the glasses.', ', the woman measures the liquor with two white cups, man begins holding to the cup an loosen it with the other arm behind his hands, then the woman gives it to the man against a wall beside an elevated metal dome.', ', the woman shows a beauty product.', ', the person adds liquid to each glass, clips them together, holds them up with the strainer, then puts them back in the glass by hand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}