{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"426a5e63-c715-5f90-af15-02853303d2d4","task_key":"default--validation--10638","task_revision_id":"1","upstream_id":"10638","short_description":"HellaSwag validation 10638","config":"default","split":"validation","body":"{\"activity_label\":\"Mooping floor\",\"ctx\":\"The man puts the mop into the blue mop bucket first and then he starts mopping. First he mops near the stove and the counter. then he\",\"endings\":\"['moves around the counter the floor sweep the points near the stove.', 'starts to cut the cake mix with knife and sharpener.', 'comes back and starts mopping in the kitchen.', 'mops on the floor that between the refrigerator and another counter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}