{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a443566c-7b22-55ce-a1dd-f061231cd5dc","task_key":"default--validation--49694","task_revision_id":"1","upstream_id":"49694","short_description":"HellaSwag validation 49694","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"A small baby sits in a car seat and hols a cup of food. The light shines bright on the right and scares the baby who begins to cry hard. the light\",\"endings\":\"['finally dazzles the baby and the video ends.', 'fades and the baby calms down and looks all around the car.', 'moves down to the left to shift back and shines on a woman standing in the sitting room.', \\\"hurts the baby's eyes as she cries and begins to wake up.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}