{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"494cd310-d37a-580c-9c3b-083a28e3c9d7","task_key":"default--validation--16718","task_revision_id":"1","upstream_id":"16718","short_description":"HellaSwag validation 16718","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"As soon as she's done a white skinned girl next to her begins her solo jump rope as the two girls on each side of her watch her. when she\",\"endings\":\"['was done a blue punctuated word appears on the screen briefly explaining the jump jump mechanics of doing it wrong and to start cutting back on.', \\\"'s done an announcer screams that they have saved the girl and exposes the girl from suspicion.\\\", \\\"'s done all of the girls start to loop wrapped she rope around and then goes back to doing their own thing twirling the rope and practicing their moves.\\\", \\\"'s finished the last white skinned girl on the left begins her solo jump rope and they all watch her until she's done.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}