{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6daf5459-47ec-5a61-858f-da108c914e26","task_key":"default--validation--24725","task_revision_id":"1","upstream_id":"24725","short_description":"HellaSwag validation 24725","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to get to montreal, canada [title] Look for an airline with regular flights to montreal. [step] Pierre-elliott trudeau international airport, located just southwest of the city, is the largest commercial airport in the area and the main thoroughfare for air travelers. Many established airlines fly here, such as air canada or united, but you may find better deals on smaller low-cost airlines.\",\"endings\":\"['[substeps] Because of their lack of miles, flying is much more affordable. When buses travel from canada to canada, bus stops get shorter and longer.', \\\"[substeps] If you're about to fly to montreal from canada, look for a major outdoor air travel agency (bus, train). These agencies charge $20 per stop, so ensure you're mobile enough to get there.\\\", '[title] Decide if you want a direct (nonstop) flight or one with multiple stops. [step] The latter option is usually cheaper, but far more inconvenient.', \\\"[title] Fly for more than 60 or 120 minutes per day. [step] If you've never been to montreal and you are not from san diego, you may have to stay overnight.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}