{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1048c6bf-b24c-5cbe-b9f4-67dd78dc5eee","task_key":"default--validation--43868","task_revision_id":"1","upstream_id":"43868","short_description":"HellaSwag validation 43868","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach your toddler about shapes [title] Teach shapes with a ball. [step] Take painter's tape and make multiple shapes on a hard floor. Call out the name of a shape and ask the child to roll the ball to that shape.\",\"endings\":\"['Everyone will have to do a variation of the shape, taking turns challenging the child and counting out their points. Take stock of the first few shapes from each variation.', 'Switch roles with your child and ask him to call out the shape names while you respond by rolling the ball to each shape he names. [substeps] You can also try this game with a beanbag.', 'Telling your toddler to roll on the floor will make them restless, but will also keep them busy. Make it soft and squishy by getting six of these shapes on the floor at first.', 'If the shape repeats itself, then the child should try to build up there first. [substeps] For example, if your toddler was drawn to a beach, say, \\\" explain the shape of the ocean.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}