{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6494f615-b58a-5fa4-8177-6e10dd397f7b","task_key":"default--validation--20910","task_revision_id":"1","upstream_id":"20910","short_description":"HellaSwag validation 20910","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to feed and hydrate a mother dog [title] Bring food to the mother dog. [step] Place food near the mother dog so that she doesn't have to leave the puppies to eat. Many new mothers don't want to leave their puppies to eat, so they abstain from eating for too long after giving birth.\",\"endings\":\"[\\\"Don't feed her her wet food, but place wet dog food in a bowl or bucket nearby. Feed it only when the mother dog seems to be getting hungry, otherwise she will refuse the food completely.\\\", \\\"When the puppies reach 2 weeks old, they become sedentary and skip eating. You should try to have snacks available for the mother dog so that she doesn't look hungry as much.\\\", 'It can seem like a strange action, but many new mother dogs take care of their puppies after birth. [substeps] Provide food very carefully when you feed a mother dog.', '[substeps] However, you can allow your dog to wait a little while to eat. Most dogs that give birth wait a few hours to eat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}