{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad90f775-5511-5784-9f88-1db172d68f11","task_key":"default--validation--5790","task_revision_id":"1","upstream_id":"5790","short_description":"HellaSwag validation 5790","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to explain autism to children [title] Don't fear using the word. [step] If you try to hide a child's autism or discuss it in whispers, they'll learn that it's something shameful or bad. You can mention the word \\\" autism \\\" in their presence even before they are old enough to understand.\",\"endings\":\"[\\\"Many parents worry a child's self-esteem will be harmed by being told their diagnosis. However, children will notice that they're different, and withholding the truth doesn't help them.\\\", \\\"[title] Tell them it is nothing to be ashamed of. [step] You can explain that it's something that you've tried and overcome before but won't now.\\\", \\\"This will make them proud and open to learning more about autism. [substeps] Pretending to be autistic doesn't work for everyone.\\\", '[substeps] Of course, it doesn\\\\'t have to be an offensive word. In fact, kids who say \\\" autism \\\" like this might think this is rude, but what they are really looking for is validation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}