{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60e05d49-b847-5e45-9ce6-5d599a78e529","task_key":"default--validation--16669","task_revision_id":"1","upstream_id":"16669","short_description":"HellaSwag validation 16669","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to make your own dog food [title] Check with your vet before switching to homemade dog food. [step] Your dog may have specific nutritional needs that your vet can explain to you. Your vet may even have some great suggestions of key ingredients that will help your pet's health.\",\"endings\":\"[\\\"If your dog's immune system is weak, put her on a diet formulated specifically for dogs. [substeps] For instance, if your vet has prediabetes or a history of oral bone growth problems, consider making your own dog food.\\\", 'Take a good look at, like, how much tofu your dog likes but a little bit more. [substeps] A large portion of tofu holds together well, particularly when combined with locally sourced protein.', '[substeps] You might ask your vet, \\\" have you tried any homemade dog food already? Do you think we could try using it on dogs? \\\" never feed a dog dry. You should let your vet know, though, that your dog\\\\'s body needs to refuel and hydrate, not changes in metabolism.', '[substeps] After you have been feeding a homemade diet to your dog for a couple of weeks take your pet to the veterinarian to make certain it is not gaining or losing weight. If weight loss or gain has occurred discuss the diet with your veterinarian to determine how your feeding should be adjusted.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}