{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c199b3d-5e35-5835-99d4-7fee8ead5dbe","task_key":"default--validation--1868","task_revision_id":"1","upstream_id":"1868","short_description":"HellaSwag validation 1868","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to recognize obsessive compulsive disorder in children [title] Don't jump to conclusions. [step] Remember that children have quirks and often go through phases that may make you wonder whether they are normal. If you're concerned that your child may have any kind of mental disorder, it's best to talk to a pediatrician or a child psychologist before you try to diagnose a disorder on your own.\",\"endings\":\"['[substeps] Your child may have borderline obsessive disorder. This may include a learning disorder, an inability to think and be articulate, or some sort of cognitive impairment.', 'Your first instinct is to freak out when your child does something that is completely out of character. Your children may be pansensitive, compulsive, or make mistakes.', \\\"If you have had your child evaluated and are still unsure, don't be afraid to get a second opinion. [title] Look out for signs of obsessions.\\\", '[title] Consider the symptoms and reactions of your child. [step] Many children with ocd have behaviors that others find unappealing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}