{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"09996376-fa3a-5bf7-8b06-d75036401b84","task_key":"default--validation--859","task_revision_id":"2","upstream_id":"859","short_description":"HellaSwag validation 859","config":"default","split":"validation","body":"{\"activity_label\":\"Using the pommel horse\",\"ctx\":\"A gymnast put power chalk power on his hands and stands near the pommel horse. Then, the gymnast approach the pommel horse to spin around many times. then\",\"endings\":\"[', the gymnast stand on his hand and jumps to land on the floor, while people applaud.', ', the gymnast put the star light on his hands and jumps high using strength.', ', the gymnast puts fluid on the pommel horse and sit on it to rise the body while position it.', ', the gymnast cross the pommel horse with the long rope while performing handsprings.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}