{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05667618-85dd-5833-b1b9-8e784ce40994","task_key":"default--validation--21950","task_revision_id":"1","upstream_id":"21950","short_description":"HellaSwag validation 21950","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy handbags on ebay [title] Search ebay for the handbag you want to buy. [step] Entering specific search criteria on ebay may help you find the exact type of handbag you want to purchase. [substeps] Go to the ebay website provided in the sources section of this article.\",\"endings\":\"['You can open the search engine on your computer at this page and navigate to the search results section. Type in \\\" handbag \\\" in the appropriate field provided then click on \\\" ok \\\" at the upper right hand side of the results list.', 'Enter keywords in the search box at the top of your session that accurately describe the handbag you are seeking. For example, if you want to locate a red purse by a specific designer, enter \\\" red, \\\" followed by the designer\\\\'s name.', \\\"Read the terms of use regarding handbag prices, coupons, delivery fees, shipping costs and future expiration dates. Find out the newspaper's trading column.\\\", 'Head to http: // ebay.com, click \\\" search for \\\" in the search bar on the upper right hand corner of the page. Here you will need a description for the handbag you want and the exact brand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}