{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e38409b-3c80-50e9-9aa0-f2c69ef4f6a8","task_key":"default--validation--14445","task_revision_id":"1","upstream_id":"14445","short_description":"HellaSwag validation 14445","config":"default","split":"validation","body":"{\"activity_label\":\"Using uneven bars\",\"ctx\":\"A man is shown talking to the camera and leads into an athletic girl swinging around on uneven bars. she\",\"endings\":\"['does this a few more times and ends by walking away.', 'hops down while the man talks again and switches back and fourth to him talking and her performing.', 'does the splits as well as lunges into the middle and falling back.', 'spins around and around while twirling her arms around and ends by throwing her arms up in the air and smiling.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}