{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"09b18505-cbac-50c2-a749-82db6c273a4a","task_key":"default--validation--29719","task_revision_id":"1","upstream_id":"29719","short_description":"HellaSwag validation 29719","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A lot of people appear in a large area, hanging out, walking and talking and the words on the screen say omaha marathon inaugural hits running festival. a large group of people\",\"endings\":\"[\\\"appear at the starting point and being running through various different scenery's while many people are cheering them on.\\\", 'are seen, taking off in roping, running and running.', 'are then shown, followed by many more on the side of road, near ice flats, which explain the university of florida logo on their dark shirts.', 'are then seen lifted in the air and men walking around dressed in clothes of the kind you would see in a chicago marathon.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}