{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"027de1c8-8c34-5d70-950b-9ed32f849b81","task_key":"default--validation--20298","task_revision_id":"1","upstream_id":"20298","short_description":"HellaSwag validation 20298","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to tell a child they are autistic [title] Tell them soon. [step] Some parents avoid telling their children about their autism because they don't want their child to feel different. However, hiding the word will not stop this.\",\"endings\":\"['Children are prone to complaining about their autism all the time. [substeps] Help your child to understand that they are being ignored by their friends.', \\\"Expressing your fear of hearing the word will start the conversation and help bridge the gap between your child and the word. [substeps] If you did something to make your child think they couldn't talk, the most important thing is to tell your child no.\\\", \\\"If your child is around other children, then they already can tell that they aren't like everyone else. If you explain autism, you can give them an explanation for this, and make it clear that this isn't their fault and they aren't bad or wrong.\\\", \\\"Parents need a day or night to talk about the condition. [substeps] Tell your child that he or she can't see anything right before bed.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}