{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db9cb778-b010-5928-9010-5d0353c46b7c","task_key":"default--validation--36267","task_revision_id":"1","upstream_id":"36267","short_description":"HellaSwag validation 36267","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to get around munich on the cheap [title] Use an automated teller machine (atm) to get the best rates for withdrawing money. [step] Try to make few withdrawals for larger amounts of money to prevent several transaction fees from adding up. [title] Pay with local currency instead of traveler's checks or credit cards as some places offering a good deal only take cash for payment.\",\"endings\":\"['[title] Use your credit card to make a few turns on the street before you take a taxi or subway. [step] Use your atm to take the bus or subway to munich.', '[step] Make a large amount out of either military advance, student-issued or visa money. [title] Take a bowl of candy available in the restaurant at the end of the day.', \\\"[step] Get some savings if you can, as you will most likely need a lot of this money when you leave. [title] Keep a record of the amount required and the type of transaction you're dealing with.\\\", \\\"[title] Purchase an airport-city-day ticket if you landed at the airport in munich. [step] It is good on all means of transportation on munich's public transportation network from the moment of purchase until 6 a.m.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}