{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46c57c2f-045e-5a07-a6a2-ce5b7a659083","task_key":"default--validation--15443","task_revision_id":"1","upstream_id":"15443","short_description":"HellaSwag validation 15443","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to help your kid overcome oldest child syndrome [title] Tell your child you are pregnant. [step] It is important that you tell your oldest child that you are pregnant with a new baby. Your child should not learn about their sibling from another family member or friend.\",\"endings\":\"['Your news about your new baby might traumatize your baby. It can be painful for your child to hear, but it is important that this information be told openly and honestly.', 'Your teenager\\\\'s child will likely find out whether you have actually told them about the baby and that they need help. [substeps] Do not say \\\" we left her to sleep.', 'Tell them that your baby is growing and will be yours. [substeps] While the news may be good for your baby, not all pregnancy announcements are good if you believe the news is good for your child.', 'Explain that your oldest child is going to become a big brother or a big sister. This will give them a feeling of importance and they can begin to think about their new special role in the family.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}