{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b64030d-3a2b-57a5-8407-f7063369a908","task_key":"default--validation--28836","task_revision_id":"1","upstream_id":"28836","short_description":"HellaSwag validation 28836","config":"default","split":"validation","body":"{\"activity_label\":\"Removing curlers\",\"ctx\":\"A girl is standing in the bathroom wearing a robe. She holds up a pink hair curler. She then demonstrates how to place the rollers in her hair. she\",\"endings\":\"['examines her hair one last time before putting a ponytail.', 'then style and twists her hair into the style it was previously in.', 'shows how she does the rollers and how they go onto her hair.', 'continues talking with two rollers in her hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}