{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32b7ac23-ceee-5151-a8ba-9a3c2b34d281","task_key":"default--validation--49953","task_revision_id":"1","upstream_id":"49953","short_description":"HellaSwag validation 49953","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to make an amazon account [title] Get started. [step] Go to amazon.com. In the upper right-hand corner, you'll see a menu marked \\\" your account.\",\"endings\":\"['\\\" click on \\\" manage account \\\" to switch this to \\\" new. \\\" from this link, you\\\\'ll be redirected to the amazon settings page.', '\\\" under the heading, you\\\\'ll see the password, passcode, and e-mail, which are all desired extensions of your account. Click on the e-mail link, which will take you to amazon.com.', '\\\" scroll down and click the \\\" account \\\" button, then choose the account you want to reserve your account. [title] Select the text field for the account you want to reserve.', '\\\" when you hover over it, a drop-down menu appears. Click on the \\\" start here \\\" link directly beneath the yellow \\\" sign in \\\" button.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}