{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0d368af5-5753-577e-b1dd-3c55edb35cf5","task_key":"default--validation--993","task_revision_id":"1","upstream_id":"993","short_description":"HellaSwag validation 993","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to call switzerland [title] Dial your country's exit code. [step] An exit code is a set or series of digits that allows your phone call to \\\" exit \\\" the country. In other words, the numbers let the telephone carrier known that the rest of the phone number should be directed outside of the country.\",\"endings\":\"['[substeps] To see a list of common exit codes, review the section on \\\" calling from specific countries. \\\" for example, the exit code for the united states is \\\" 011.', '[substeps] For example, a phone call to switzerland from 0800 would be 0800. [title] Dial the country code from 1 of the countries listed.', 'You can generally find this somewhere along the dial pad on all the cell phones. [substeps] The exit code on the us phone will be 93.', 'Cell phone service stations can also be easily forgotten. [substeps] Dial the exit code for most countries within switzerland.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}