{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed9e6ade-c65a-5fda-81a3-cf8dd7489bb3","task_key":"default--validation--20429","task_revision_id":"1","upstream_id":"20429","short_description":"HellaSwag validation 20429","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"The woman swaddles the baby in a white blanket and then begins to apply water soaked cotton balls, from a nearby water filled container, to the baby's eyes and nose and face. the woman\",\"endings\":\"['finishes and talks to the camera again.', \\\"wipes the baby's eyes, then she uses a cloth to move the contact lens lens cover.\\\", \\\"wipes the baby with her striped towel then inserts more cotton balls into the baby's eyes.\\\", 'applies several black colored liquid and the baby instantly begins to smile and cry.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}