{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"404b11f3-f19e-5aa2-8afd-62562a66d438","task_key":"default--validation--20438","task_revision_id":"1","upstream_id":"20438","short_description":"HellaSwag validation 20438","config":"default","split":"validation","body":"{\"activity_label\":\"Gargling mouthwash\",\"ctx\":\"The girl then takes the mouthwash and starts to put it over head and playing with is while making funny faces. after\",\"endings\":\"[\\\"she's done putting the mouthwash in her mouth, gets sideways, splashes some water on her legs, rinses she mouthwash very messily, shakes the mouthwash into her mouth and spits it in the trash bin.\\\", ', she sprays water all over her face and repeats the actions.', 'taking about it, the girls takes a gulp of it and starts to gargle it and holds it in her mouth.', \\\", she tries to get it off her head but it won't go down, then she tries to get it down between her legs but can't and when she tries to walk she falls down.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}