{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"adb250d8-6bf9-555f-86e0-aa6d0abf30f6","task_key":"default--validation--25653","task_revision_id":"1","upstream_id":"25653","short_description":"HellaSwag validation 25653","config":"default","split":"validation","body":"{\"activity_label\":\"Playing ice hockey\",\"ctx\":\"A man wearing a suit is sitting behind a desk with papers in front of him, a blue screen behind him, and he's talking into the camera while words scroll at the bottom. video clips\",\"endings\":\"['of hockey players are displayed in taking turns and shifting across the screen without really using a device.', \\\"are shown of the home he's renovating and steps are shown on the stairs as well as the roof, the roof, teams lined up, and people wearing signs talking.\\\", 'are shown of people performing flips on the same floor where people pass, and shows them performing several flips before the man walks back the way he came.', 'of hockey players on a rink and people in the stands begin to play while words are still scrolling on the bottom, and at times a man is shown talking to the camera with blue words on the bottom that say \\\" amateur ice hockey league crowned with gala game against past legends \\\".']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}