{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c14e80e-13c1-5b4c-9e15-e3e9afabdbe4","task_key":"default--validation--28261","task_revision_id":"1","upstream_id":"28261","short_description":"HellaSwag validation 28261","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to make the most of the night life when traveling [title] Research events ahead of time. [step] Before you leave for your trip, it may be best to look up information about where you are staying. Picking out events that sound interesting and fun before the plane flies can save you and your group plenty of hassle.\",\"endings\":\"['[substeps] Online searches are a great place to start for any vacation. Start with a broad search query such as \\\" things to do in paris \\\" to get a feel for what there is to do.', \\\"If you're traveling alone, inquire about travel guide websites, catalogs, travel publications, and other information. [substeps] It may help to have some music on you so that you can listen to it a lot while you travel.\\\", \\\"[substeps] If this is a little trip you have probably already planned, there's no need to wait around wondering when to arrive. Because flight attendants can predict few things ahead of time, they may be able to help coordinate last minute events that might be something more important.\\\", \\\"Look on websites like guidebooks, local news sites or airfare websites to get an idea of everything you'll need. [substeps] Try to pick an event that isn't heavily publicized.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}