{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5d6da509-3085-5f5e-8278-286c02b0bfd3","task_key":"default--validation--22930","task_revision_id":"1","upstream_id":"22930","short_description":"HellaSwag validation 22930","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to strip cloth diapers by boiling [title] Remove any solid waste. [step] Newborn diapers probably won't have any removable waste, since breast milk stools are water soluble and formula stools are too fluid to shake off. Older babies and toddlers on solid foods will have more solid stools, however.\",\"endings\":\"[\\\"You should remove any found waste or soiled stool at least 50 minutes before baby meals to eliminate it. [substeps] Wash the baby's hands first to prevent contamination of food.\\\", 'Shake these stools off into the toilet before washing and stripping. [title] Wash the diapers in a washing machine.', \\\"If the mother breastfeeds her toddler or things get messy while they're using formula, you can remove any stool yourself. [substeps] Wash cloth diapers in the hot water pail or in a separate compartment to prevent them from getting damp.\\\", 'Milk not taken out will still be sticky when you remove it from the package. [substeps] You may need to scrape away any residue with a box cutter or a sponge.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}