{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39ac3268-cb35-57f7-8654-4fa5d90018b8","task_key":"default--validation--30131","task_revision_id":"1","upstream_id":"30131","short_description":"HellaSwag validation 30131","config":"default","split":"validation","body":"{\"activity_label\":\"Cheerleading\",\"ctx\":\"A large group of cheerleaders are seen walking out on a stage while people watch on the sidelines. The group begins performing a cheer routine while jumping and flipping all along the stage. the group\",\"endings\":\"['continue quickly as the girls keep jumping around on the stage and ends by stepping up and walking away.', 'continues performing until one scene captures their small group in the end.', 'continues cheering and ends by holding a finishing pose.', 'continues their dance routine and ends with them dressing up in colorful shirts and holding a pose.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}