{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a8d6f079-1b8f-5705-985c-1ea1ffa081c9","task_key":"default--validation--43672","task_revision_id":"1","upstream_id":"43672","short_description":"HellaSwag validation 43672","config":"default","split":"validation","body":"{\"activity_label\":\"Baton twirling\",\"ctx\":\"The girls spins repeatedly and catches her baton. The girl spins and two batons roll in and she grabs each. the girl\",\"endings\":\"['throws the baton down and spins the baton around her body.', 'spins again and the other one moves and falls.', 'spins, catches her baton and poses for her finish as the crowd claps.', 'forgets the baton spins and lands on her bedroom floor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}