{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"21971dad-49ce-5389-a2f6-f0bb7ad15ac0","task_key":"default--validation--20411","task_revision_id":"1","upstream_id":"20411","short_description":"HellaSwag validation 20411","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to buy books for your kindle [title] Go to the home screen on your kindle tablet. [step] Do so by pressing the \\\" home \\\" button at the bottom of the screen. [substeps] Your kindle must be connected to the internet and linked to your amazon account.\",\"endings\":\"['This means it will not get started until you visit http: // www.amazon.com/download-your-book-for-kindle-right-when you log in. [title] Go to the book store on your kindle.', '[title] See what other books you want to review. [step] In the \\\" books \\\" window, hover your cursor over the interest or hobbies section to see a list of other interest books going through.', 'For convenience, you may want to turn on 1-click ordering. [title] Tap books at the top of the screen.', 'In order to file a new file, your kindle must be linked to your amazon account. From here on, you have to log into and of course your amazon account too, or your kindle will not be used.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}