{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"730ba257-a460-5152-9e78-0af6aabef831","task_key":"default--validation--4847","task_revision_id":"1","upstream_id":"4847","short_description":"HellaSwag validation 4847","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to delete an account on the knot [title] Log on to your account. [step] To begin the process of deleting your account, go to theknot.com and hit the \\\" log in \\\" link. This link is located under the blue \\\" sign up \\\" button on the knot's main page.\",\"endings\":\"['Enter your email address and password in the fields provided then click \\\" continue. \\\" [title] Click on \\\" account settings \\\" if you\\\\'re not already logged in.', 'Once you\\\\'re in, click the \\\" sign up \\\" button below the main page explaining that you want to file an account with the knot. A confirmation message will appear, indicating that \\\" you\\\\'ve changed your password \\\" is enabled on your account.', '[substeps] If you\\\\'re already logged in, go straight to the \\\" account settings \\\" link at the far right of the top menu bar. [title] Save any information you don\\\\'t want to lose.', \\\"This link is, along with its restrictions, the most important thing for you to know. [substeps] If you don't already have a login account, follow the on-screen instructions to create one.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}