{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e759c1d1-d91e-5078-913b-49648d70dc6c","task_key":"default--validation--45066","task_revision_id":"1","upstream_id":"45066","short_description":"HellaSwag validation 45066","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to sign up for yahoo [title] Visit the yahoo homepage. [step] When you're ready to begin, click \\\" sign in \\\". This will take you to the login screen, where you'll be able to register a new account.\",\"endings\":\"['[substeps] You will then be directed to a page where you can register your yahoo id, microsoft email address, gmail id, and password. Click \\\" sign up \\\" to load your yahoo id, and enter the information given on the screen.', '[title] On the next page, click \\\" create new account \\\". [step] Clicking on the \\\" sign in \\\" link should take you to a screen where you\\\\'ll have the option to log into an existing account or create a new one.', '[substeps] If you are already logged in to yahoo, skip to step 5, because yahoo may ask you to create an account here. If you have already created an account, enter your yahoo id information on the pop-up, and tap sign in.', 'You can also create an account on a whim. You can create one for free or for a fixed sum, depending on what you want.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}