{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b26cf75f-eb99-5404-91da-c2b90130f045","task_key":"default--validation--14551","task_revision_id":"1","upstream_id":"14551","short_description":"HellaSwag validation 14551","config":"default","split":"validation","body":"{\"activity_label\":\"Hitting a pinata\",\"ctx\":\"The boy is standing under the tree with blindfold thing to hit the pinata but missed it. a man\",\"endings\":\"['is showing the pinata the kid hit and he pulls the stick out until it hits an orange pinata.', 'came up to him and assist him, the boy hit the pinata once.', \\\"put a piece of blindfolds on the boy and he tries to hit it with his hand and can't find it.\\\", 'is kneeling in front of the boy in the room.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}