{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8942d77c-b8c4-5ffd-ae17-69577943d052","task_key":"default--validation--12028","task_revision_id":"1","upstream_id":"12028","short_description":"HellaSwag validation 12028","config":"default","split":"validation","body":"{\"activity_label\":\"Wakeboarding\",\"ctx\":\"Then, a woman and a man water ski doing acrobatic jumps. A boat sails empty in the river. After, men water ski jumping and turning around. next\",\"endings\":\"[', a woman is standing next to an ocean and the man and woman water ski.', ', the boat slows down and the woman and man fall on the rock surface.', ', a person surf on the waves created by the boat, after the man water ski jumping and flipping high.', ', more people take off their clothing and do half jumps in the river.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}