{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f960275e-af01-52b9-8427-6bca2e9c9a12","task_key":"default--validation--17190","task_revision_id":"1","upstream_id":"17190","short_description":"HellaSwag validation 17190","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for a new navel piercing [title] Get your navel professionally pierced. [step] Do your research to find a reputable piercing shop with trained, professional piercers. You can ask friends and family about the places they got their piercing done, and ask if they would recommend the place.\",\"endings\":\"['You can also talk to your piercer if you want to get a piercing either in the navel area or almost anywhere else on your body. They can also tell you about the comfort, comfort, condition, risks and penalties that come with piercing piercings.', \\\"You never want to skimp on the quality of the establishment or piercer you visit. The more professional the business and certified its employees, the less likely you'll experience problems or infections with your piercing.\\\", \\\"[substeps] You should only get an piercing done where you are a trained piercer and pay special attention to your navel. Piercers will tell you if you don't get your piercing done.\\\", 'Piercing shops are usually expensive and will probably charge more. [substeps] Ensure the piercer you visit is reputable.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}