{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fff868e5-10b5-5aee-8bb8-5e5ab46f35a6","task_key":"default--validation--43950","task_revision_id":"1","upstream_id":"43950","short_description":"HellaSwag validation 43950","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A grey bunny is standing on a bed on a black towel eating something in his hand. As he eats, the bunny begins to grabs his ears and shakes its head. struggling more\",\"endings\":\"[', the bunny reaches for his fur and the object flies off into the distance.', ', the bunny finally gets sucked out in a long rope.', ', the poodle stops and turns his head around to be let down and eventually pick the cat up and continues the grooming.', ', the bunny bends down and keeps eating something before looking out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}