{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0d78f2b-4cac-5aab-80bb-04bf85ed040f","task_key":"default--validation--2288","task_revision_id":"1","upstream_id":"2288","short_description":"HellaSwag validation 2288","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to call austria [title] Enter the international exit code to dial out of the country. [step] All countries require you to enter an exit code to dial out, however, this number can vary depending on the country you are calling from. [substeps] For instance, to call from the us or canada, dial \\\" 011 \\\".\",\"endings\":\"['Any country within europe and within canada may also use the exit code \\\" 00 \\\" as long as you dial in norwegian. Unless you\\\\'re planning to play some sort of game like flag fighting, you can only use 00.', 'While you can dial \\\" 011 \\\" with any country code, each country uses its own. For instance, in the u.k., dial \\\" 011 \\\" if you are calling from canada.', 'Other countries may use \\\" 0 \\\" or \\\" 11 \\\". A quick internet search for \\\" calling abroad \\\" or \\\" international exit codes \\\" will give you the exit code for your country.', 'Most countries in the uk and abroad require you to use the us exit code if you are calling from canada. The area code for the united states, for instance, will be \\\" 011.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}