{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d4ce6dd-0b92-5f9f-bd10-63537d4a0af0","task_key":"default--validation--40951","task_revision_id":"1","upstream_id":"40951","short_description":"HellaSwag validation 40951","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to get universal studios tickets from costco [title] Search for tickets on costco's main website. [step] Navigate to www.costco.com. In the search bar at the top, type in \\\" universal studios.\",\"endings\":\"['\\\" provide all the necessary information about your interest in an idea. [substeps] You can find the login code for costco on the website.', '\\\" scroll down in the results until you find the right search engine. To access the site from the main website, click on it.', '\\\" the search results will bring up any universal studios tickets if they are currently for sale, though costco doesn\\\\'t always have these tickets in stock. [substeps] Alternatively, navigate to https: // www.costco.com/theme-parks.html and look for universal studios tickets on that page.', '\\\" your best bet is to look on the left column. Click on the link that reads \\\" buy tickets to costco\\\\'s brazil club downtown , 500 800 costco avenue.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}