{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2833b21d-9146-5350-a679-6be525815e6e","task_key":"default--validation--43857","task_revision_id":"1","upstream_id":"43857","short_description":"HellaSwag validation 43857","config":"default","split":"validation","body":"{\"activity_label\":\"Plataform diving\",\"ctx\":\"A man in a black speedo is standing on a diving board with his hand down on his sides. He then begins running and does two hops to get to the end of the board. finally\",\"endings\":\"['he jumps down and is shown again at the bottom.', 'he goes back down as he gives a thumbs up.', ', he is at the end of the diving board and does a flip into the water as the crowd on the side watches.', 'he takes off and does three belly kicks to wind up before making it to the end, and there shorts has been thrown from the beginning of the board.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}