{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a74ebee7-65de-5156-889e-17829e2abd3c","task_key":"default--validation--50045","task_revision_id":"1","upstream_id":"50045","short_description":"HellaSwag validation 50045","config":"default","split":"validation","body":"{\"activity_label\":\"Canoeing\",\"ctx\":\"Names are shown on a list and leads into a person riding down a river on a kayak while others watch. more people\",\"endings\":\"['are seen riding down a river on tubes while one man continues riding backwards and another riding alone on a line.', 'are seen riding down on kayaks in rafts and look away.', 'fall off the kayak as the boat chases them.', 'are seen riding down the river and ends with people sitting inside at tables and smiling to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}