{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13409d4a-8087-5279-8cd8-8c30b137faaf","task_key":"default--validation--8106","task_revision_id":"1","upstream_id":"8106","short_description":"HellaSwag validation 8106","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to raise kids with dogs [title] Allow your dog to enter the nursery. [step] This will help your dog become accustomed to new furniture, new smells, and a new environment. [title] Apply baby products to yourself or your dog's toys.\",\"endings\":\"[\\\"[step] Baby products come in a range of scents, including vanilla, milk, baby shampoo, and pet food, and these products are readily available at many drugstores, drug stores, and pet supply stores. Apply baby products to your dog's toys once a week after exercising, while they are dry and warm in the clinic.\\\", \\\"[step] The smells of baby lotion or baby shampoo will allow your dog to become familiar with your baby's smell before they arrive in your home. [title] Play tapes or videos of babies crying on a regular basis.\\\", '[step] It is best for your dog to go to the hospital as soon as possible because adoption will not happen immediately. [title] Keep your dog isolated in the hospital.', '[step] Spraying hairspray throughout your home-a household product recommended above-is a great way to reinforce these smells, but can also help relax your dog. [substeps] Some people find that having babyproofed their homes with babyproof products helps discourage their hyper behavior.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}