{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"598ec9f9-b180-5ef0-8af5-b089171b99ef","task_key":"default--validation--23142","task_revision_id":"1","upstream_id":"23142","short_description":"HellaSwag validation 23142","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to privatize your yelp bookmarks [title] Open your web browser. [title] Visit the yelp homepage. [title] Login to your account, if you haven't done so already.\",\"endings\":\"['[title] Click the button marked \\\" space \\\". [step] This will bring up a page showing your browser\\\\'s history.', '[step] You should see directly above that a line that begins with \\\" public-others can see your bookmarks \\\". Directly to the right of this you should see the link for making your bookmarks private.', '[step] You need to have a facebook account for this. [substeps] If you don\\\\'t have a facebook account, then you can save some of your entries and sign up for an account by typing \\\" yelp \\\" in the search field at the top of the screen.', \\\"[title] Follow the on-screen instructions to begin the process. [step] You can also follow the out-of-order confirmation message if one of your account's pages has been blocked or if you can't find the newest version of the app that contains your bookmarks.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}