{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a3639d4f-732a-57a9-84a5-2859b599d8c9","task_key":"default--validation--37909","task_revision_id":"1","upstream_id":"37909","short_description":"HellaSwag validation 37909","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to make international calls from the usa [title] Dial \\\" 011 \\\" on your keypad. [step] Use the international direct dialing (idd) number for the u.s. before dialing any other numbers. This signals that the phone number you will dial next is outside of the united states.\",\"endings\":\"['[substeps] The idd number is a voip code for the us. Some countries, like mexico and costa rica, share the idd code with another country, though the country code for that region also is b3.', '[substeps] Remember that \\\" 011 \\\" is the prefix for only the united states. If you are placing a call from a country other than the u.s., you will need to find the idd for that country.', '[title] Dial (1) 877-1213-2580. [step] This number is the same as the number 1 511, and each country has an idd code.', '[substeps] If you cannot dial the uk on 011 (idd), the number you dial from the united states directly should be delivered to the blackberry address, isp/mse/product code, in english and spanish. Any foreign exchange country that does not use an idd line.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}