{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43e5dd45-f952-58c2-8db6-5c9c60e9a2fa","task_key":"default--validation--29718","task_revision_id":"1","upstream_id":"29718","short_description":"HellaSwag validation 29718","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A colorful intro appears and in includes a logo and the words \\\"introducing running festivals a distance for everyone!\\\". a lot of people\",\"endings\":\"['from different seasons are present as they enjoy the tourist events in the forum.', 'are gathered about skating down street, performers are playing cars, divers walk up street and are assisting each other.', 'are seen looking around and cameras showing individuals running and a little boy in a sink.', 'appear in a large area, hanging out, walking and talking and the words on the screen say omaha marathon inaugural hits running festival.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}