{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c5c63de1-455d-5da6-ad73-6ee1db4dbc06","task_key":"default--validation--30495","task_revision_id":"1","upstream_id":"30495","short_description":"HellaSwag validation 30495","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"A baby girl is sitting next to all of her toys eating an ice cream very messily. Her mom takes the ice cream away from her and she starts to cry . her mom\",\"endings\":\"['starts laughing and starts fighting with her later and ends by taking her show and the baby is smiling as she lights a new cigarette.', 'watches her while she eats while bells on the ice cream displays in smaller screen, and then fades.', 'begins messing with the camera lens in her right eye.', 'gives her back the ice cream and she is happy again, back to eating it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}