{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5d165f3-6e98-5e2f-9a42-fae9f796260b","task_key":"default--validation--40098","task_revision_id":"1","upstream_id":"40098","short_description":"HellaSwag validation 40098","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to securely convert paper savings bonds to electronic securities [title] Gather together all your paper us savings bonds. [title] Visit the treasurydirect webpage. [title] Login to your treasurydirect account.\",\"endings\":\"['[title] Locate the toolbar that contains most of the buttons and click the \\\" managedirect (r) \\\" link. [step] It\\\\'ll be just underneath the treasurydirect icon.', '[title] Look for your savings bonds listed in the \\\" i \\\" section of the page. [step] This will show the treasurydirect link box inside the lower right hand corner of each bond bond, and near the top.', '[title] View your online savings bonds program (cordia). [step] Your account at cordia is, essentially, an e-commerce website, containing summaries of your bank statements, news articles, brochures, and other content.', '[title] Click save once to save all your electronic money. [title] Locate the account in the \\\" treasury & refinancing \\\" section.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}