{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9530cc93-224a-54eb-9e83-abdbc62495be","task_key":"default--validation--7067","task_revision_id":"1","upstream_id":"7067","short_description":"HellaSwag validation 7067","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to call australia from the usa [title] Locate the city code of the city you are calling. [step] In australia, each city has an individual city code. You will need to dial the area code during the calling process.\",\"endings\":\"['[substeps] For example, if you are calling australia from the united states , 115 is the city code of 1 , 149. As such, the city code would fall within this 15 digit area code.', 'For example, you will dial the sydney area code. [substeps] If you are calling australia from a landline cell phone, dial the area code of the cell phone sitting at the bottom left corner of the screen.', 'Different countries have different codes, but generally your start call priority code will be 041. If calling from a landline, there is an option to dial 041.', \\\"You should be able to locate the area online with a simple google search. [substeps] For example, melbourne's city code is 3.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}