{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff8ef531-8981-5cc9-9f23-309e4ebd5eef","task_key":"default--validation--39991","task_revision_id":"1","upstream_id":"39991","short_description":"HellaSwag validation 39991","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"We then see people lined up a the start of a a marathon take off running as words stream at the bottom of the screen. Asian script appears and a man stops running to look at the camera. the man\",\"endings\":\"['throws money on the ground then taunts someone.', 'then is shown running in a marathon while knees slightly bent from running on his waxed behind.', 'runs off the marathon course and we then see shots of various location such as lakes, weddings, cities the ocean.', 'throws the sword away and we see ending credits with the title at the bottom.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}