{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7db38919-bf0a-54ac-a822-28232555233d","task_key":"default--validation--16716","task_revision_id":"1","upstream_id":"16716","short_description":"HellaSwag validation 16716","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"There's a lot of children and adults in a large indoor gymnasium with a wood floors and some are rope jumping while others are watching while sitting or standing. a brown skinned girl in the forefront\",\"endings\":\"[\\\"isn't wearing a shirt as she looks around and watches another class but continues to watch the other children gymnasts.\\\", 'focuses on the rope climbing instructions on her phone.', 'begins to jump rope very quickly.', 'of the group leaves the gym and is followed by another child jump rope.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}