{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"ea3749dd-3b68-58f7-b81a-4ac4a06f7561","task_key":"default--validation--858","task_revision_id":"2","upstream_id":"858","short_description":"HellaSwag validation 858","config":"default","split":"validation","body":"{\"activity_label\":\"Using the pommel horse\",\"ctx\":\"A gymnast put power chalk power on his hands and stands near the pommel horse. then\",\"endings\":\"[', the gymnast put strength and power item on his body and falls.', ', the gymnast approach the pommel horse to spin around many times.', ', the gymnast blade the board and stands in position facing the pommel horse doing blade moves.', ', the gymnast moves around and flips the chalk power to full weight and continues gymnastics for the next competition, after he turns and stands near the pommel horse.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}