{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0542ddfc-e8ae-59e9-b578-e7819d01a612","task_key":"default--validation--9072","task_revision_id":"1","upstream_id":"9072","short_description":"HellaSwag validation 9072","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach a blind or visually impaired child to tie their shoes [title] Start giving lessons between ages five and seven. [step] Teachers of the blind and visually impaired often stress the importance of learning age-appropriate skills, which contributes to developing motor functioning and increasing independence. Most children learn to tie their shoes between the ages of five and seven, so a blind child should also start learning during this age range.\",\"endings\":\"[\\\"[substeps] By age five, the child should be able to tie their shoes easily and effortlessly. They do not need to grab the laces or add more pressure to them after they've finished tying the shoes.\\\", '[title] Know that having young children learn to tie their shoes may take a considerable amount of time. [step] It may involve more resorting to traditional 1-step training for children who do not have the ability to walk.', \\\"[substeps] Start off slow with the child, since learning this skill won't happen overnight. You don't have to start until you feel they're ready.\\\", '[title] Gain a mastery of hand-eye coordination. [step] Dementia is often associated with memory loss, which can cause deep nerve damage.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}