{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2cd936d-38c8-5fd1-8a65-d6fd5bce3e3f","task_key":"default--validation--27212","task_revision_id":"1","upstream_id":"27212","short_description":"HellaSwag validation 27212","config":"default","split":"validation","body":"{\"activity_label\":\"Discus throw\",\"ctx\":\"A man with a blue shirt, blue sweat pants and green socks is standing outside in a large field walking into a circle of cement. Once the guy is on the circle, he begins walking around the circle to find his perfect spot. everything\",\"endings\":\"['starts to look very interesting as he begins to bump into the wall and fall into it.', 'is as she was gone, she was standing then spinning around a few times.', 'is in place as he grabs the end of the hopscotch stick with his foot and uses it to move towards the outline of the jump.', 'is perfect and then he throws the discus that was in his hand into the field as he is spinning around.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}