{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3006e91-19cc-5204-9caf-63ba06b560ae","task_key":"default--validation--22811","task_revision_id":"1","upstream_id":"22811","short_description":"HellaSwag validation 22811","config":"default","split":"validation","body":"{\"activity_label\":\"Tai chi\",\"ctx\":\"A man is seen standing in a field with his eye closed and standing still. The man then begins moving his arms and legs around his body. he\",\"endings\":\"['then uses his hands to move the objects around and then cuts it all to pieces.', 'is seen in several angles swinging around himself and others as well.', 'continues moving his body around with his eyes closed.', 'moves himself all around the field and ends by brushing off the dirt with his hand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}