{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef91c1fe-de1e-595b-9bf5-0201aadb285b","task_key":"default--validation--6358","task_revision_id":"1","upstream_id":"6358","short_description":"HellaSwag validation 6358","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get your nipples pierced [title] Find a reputable piercing/tattoo parlor in your area. [step] Is the first and arguably most important step in getting your nipple pierced. Piercing establishments are often found in urban areas, but you'll come across them in even the remotest of places.\",\"endings\":\"['Finding a reputable piercer is always a key element. [substeps] Dentists and jewelers are crucial people in a pierced piercing business.', \\\"[substeps] Once you have found a reputable piercing parlor, ask to speak to an employee to get a recommendation and get your tattoos noticed. It's always a good idea to look at a tattoo parlor's website or to speak to an experienced piercer.\\\", 'Google, as with most things, is a good way to start your search for a parlor. The website should be professional and well-organized, with plenty of helpful information.', '[substeps] Places to shop for your nipple piercing include drugstores, fashion stores, and tattoo studios. You will be able to find someone who specializes in body piercings, ear piercings, and intrauterus piercings.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}