{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f245d1e2-a441-5224-a6ca-e0c6d09b9338","task_key":"default--validation--12547","task_revision_id":"1","upstream_id":"12547","short_description":"HellaSwag validation 12547","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to calculate exchange rate [title] Estimate the amount of money you wish to exchange. [step] Think about how much money you're budgeting for the trip. Or, if you know how much money you'll need in the other country, work backwards and start with the foreign currency.\",\"endings\":\"['[title] Look up the exchange rate of the currency to which you wish to convert. [step] You can find this information on a google search, or on several banking or financial websites.', \\\"[substeps] For example, assume you have $700 million in the us dollars. Imagine you're budgeting $600 million in the united states dollars.\\\", 'That should show how much change you may charge for your trip. The united nations international exchange rate is 60 cents a dollar and is the currency (a) of the country.', 'Some exchange-rate accounting software supports this mathematical study. Some agree that with these large numbers of possible exchange rate exchanges you can either trade by currency or one of the us currencies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}