{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2911b03f-4dec-52cc-8e8f-96658e195f27","task_key":"default--validation--42556","task_revision_id":"1","upstream_id":"42556","short_description":"HellaSwag validation 42556","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to avoid getting lost on a college campus [title] Book a campus tour. [step] Contact the college admissions office and request a date and time to tour the campus. [substeps] If you have already had a campus tour, perhaps before deciding to enroll, there's no harm in doing another.\",\"endings\":\"['You can book your tour shortly after college if you choose to do it at all. If you take tours only within a specific timeframe, you will be more susceptible to lost-spot lodging charges.', 'This tour would be more helpful in really getting to know the area and asking more specific questions of the staff, such as the best routes to take from the dorms or how long it takes to get to the cafeteria from the library. [title] Download a map of the college or university.', \\\"As you stroll or even drive to campus, you'll be able to find out more about the various attractions, architectural developments, buildings, and neighborhoods. However, if you have to stop somewhere to see a certain exhibit, the time is probably not worth the cost.\\\", \\\"There are bound to be free tours available. If you have a group of students and are still looking for a place to stay, there is probably a good chance you'll save money on student accommodation.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}