{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f074aaa-1cac-5707-afa9-24134b279a76","task_key":"default--validation--24759","task_revision_id":"1","upstream_id":"24759","short_description":"HellaSwag validation 24759","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to complete an i9 form [title] Put your full legal name on the form. [step] Include both names if you have 2 first names or 2 last names. You only need to use your middle initial, not your full middle name.\",\"endings\":\"['Don\\\\'t abbreviate any of the letters or numbers, don\\\\'t put \\\" m \\\" for \\\" us \\\" in the top right corner, and don\\\\'t include your e in the top right corner. [title] Pin the header of the paper to the subject (s) of the form.', 'You\\\\'ll also need to add any other names you\\\\'ve had, such as your maiden name, if you have one. [substeps] If you happen to only have 1 name, place it in the last name field and \\\" unknown \\\" in the first name field.', '[title] Write down the subject and the time it was issued. [step] Write down what world it was issued in (if it was issued in the united states) and what branch or country it was issued in, and what language it is written in.', 'If you want to include your general city and zip code, you can\\\\'t do so. [substeps] If you are using itunes, click the \\\" general \\\" button in the left pane of the form under the file menu.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}