{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57a1d25b-b903-55ae-84c8-7d2c59cf0d69","task_key":"default--validation--30054--variant-1ba100b2fe50","task_revision_id":"1","upstream_id":"30054--variant-1ba100b2fe50","short_description":"HellaSwag validation 30054","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to buy discontinued products [title] Check with the brand. [step] The manufacturer of the item might keep a discontinued line for future sales or new introductions to the market. [substeps] Look for customer service information on the brand's website.\",\"endings\":\"[\\\"In addition to identifying the manufacturer of the product, consumers should also be able to find information about their product and any expiration dates. [title] Consider the packaging you'll take with you.\\\", 'If you can\\\\'t find any information from other sellers, ask for it directly. Check the box where the \\\" i \\\" document should appear and make sure the product you want has a discontinued line.', 'This also might tell you a little about how often an item is exported. If it was sold in a brand other than the one you used, you may be able to borrow it, but if it is owned by another brand, you can only borrow it.', \\\"For example, if your favorite pair of nike sneakers has been discontinued, you can contact nike to see if they provide a way for customers to purchase them directly. Search a company's website if you cannot find the discontinued product in stores.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}