{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"459aa760-674d-5b3b-a878-7b63cb2ef4e3","task_key":"default--validation--15717","task_revision_id":"1","upstream_id":"15717","short_description":"HellaSwag validation 15717","config":"default","split":"validation","body":"{\"activity_label\":\"Doing crunches\",\"ctx\":\"First he shows viewers how to properly do a crunch and his feet are apart, arms on his chest, and each time he goes up the elbows touch his legs. when he\",\"endings\":\"['gets to the end he faces right at the camera.', \\\"goes back he's smiling and holding his arms up stretching his body and looking into the camera.\\\", 'finishes his basic crunch he is very pleased that his technique is finally working as the two men lead with a \\\" ooooo \\\".', \\\"'s done, he stands up and tells viewers more helpful information about how to have a good workout.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}