{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6f7223b-1193-50e6-b8f2-bf60eefef034","task_key":"default--validation--9864","task_revision_id":"1","upstream_id":"9864","short_description":"HellaSwag validation 9864","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A brown rabbit with a white belly is sitting on the ground and grooming itself while licking its right side. when the rabbit\",\"endings\":\"['is released, it runs to the toilet, and those on the floor get it.', 'stops licking its right side it briefly faces forward and its left back foot scratches its left ear then he licks his left paw.', 'is done eating, the owner shows off its teeth as it licks the owner finger in its mouth.', 'is finished licking its paw, the dried off part of its fur and off of its black nails.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}