{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c08da133-7f94-5889-a110-4bf8eb8b9d3f","task_key":"default--validation--31497","task_revision_id":"1","upstream_id":"31497","short_description":"HellaSwag validation 31497","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to influence your baby's first word [title] Focus on simple words with repeating syllables. [step] A baby's first word is normally something that is important to them, such as \\\" mama \\\" or \\\" dada \\\". Most babies' words are repeated syllables and are fairly simple words.\",\"endings\":\"['There is no noticeable difference between them, but there is one of both the different stage of language. [substeps] A c stands for pinky-tip.', 'Keep this in mind when trying to influence your baby\\\\'s first word. [substeps] To say a real word, they must associate it with something-a baby won\\\\'t say \\\" carrot \\\" as his first word because he probably doesn\\\\'t know what a carrot is.', '[substeps] Identify the beginning of any word and then repeat out loud that phrase \\\" mama \\\". Think about saying \\\" siri \\\" instead of \\\" doll \\\".', \\\"Even if it sounds like it's being repeated multiple times in different ways, it is important to focus on words that represent the changes your baby is going through. [substeps] For example, keep your baby's first word simple and short.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}