{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bf10140-3a5f-5b46-b596-0149cd80b696","task_key":"default--validation--8237","task_revision_id":"1","upstream_id":"8237","short_description":"HellaSwag validation 8237","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to toddlerproof your bedroom [title] Push your bed into a corner. [step] Toddlers love to stand up on the bed, and having two walls near your bed helps to reduce the possibility of your toddler falling off. [title] Check to see if your bed as a black-hole between the headboard and the mattress where your pillows go.\",\"endings\":\"['[step] If so, then your bed is a black hole. The more sleep your toddler gets, the bigger your black hole will be.', '[step] If your bed has this, put something in there to prevent the potential for your toddler to get stuck in this gap. It will also stop things from dropping behind the bed, such as the sippy cup or a bottle.', '[step] If so, set your bed against a wall or in the corner. [title] If your bed can fit two or three games, and your child can climb on it easily, consider a bed that will only fit one game.', '[step] Imagine an uncomfortable, closed-up space between the headboard and bed. Use a black-hole as a closet door with a black velcro tag on the top.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}