{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a09365bf-cc0a-5862-8280-5148fabbedf7","task_key":"default--validation--5964","task_revision_id":"1","upstream_id":"5964","short_description":"HellaSwag validation 5964","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to travel to isla mujeres [title] Get to cancun. [step] Though isla mujeres has a very small private planes airport, the usual method for tourists is to fly into cancun, mexico, and then catch the high-speed ferry over to the island. Flights and hotels can be booked through your local travel agent, or for additional savings, one of the premier online travel websites.\",\"endings\":\"['[substeps] The car rental option is out for travelers who can live for a month or more while travelling to cancun. It can be cheaper to purchase in cancun than to rent a car in california.', 'At cancun private planes are also another main option and are usually free. [title] Learn about the customs and guidelines regarding traveling to isla mujeres.', \\\"[title] Take the ferry to isla mujeres. [step] Originally, son's of local fisherman would take small launches over to the mainland and pick up visitors for their stay on the island.\\\", '[title] Inspect the village you plan to visit. [step] Cruz harapon is located across the area you expect to be flying into later in the day, and it has unsalted palm trees and a great view.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}