{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1ff7093-15c4-5696-904c-883bd19f200a","task_key":"default--validation--30231","task_revision_id":"1","upstream_id":"30231","short_description":"HellaSwag validation 30231","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to redirect a page in english wikipedia [title] Make sure you follow the wikipedia guidelines on redirecting in english wikipedia. [title] Create an account. [step] Otherwise you will not be able to create a new page.\",\"endings\":\"['[title] Log in if you are not already logged in. [title] Go to the page you want to redirect.', '[title] Get the information in the first letter. [step] This is the first letter of a foreign language and should be used whenever you encounter a foreign word or the form of that foreign word.', '[substeps] Create an email address for your page. If you had previously sent an email in english, choose \\\" [insert your article id] \\\" (or \\\" google \\\").', 'Be very diligent: people really hate wikipedia because it uses ungrammatical content that they find too disgusting. But you can, as it says, have much fun researching in wikipedia if you have the skills.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}