{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9394f5c-fb18-5b1f-96e6-cc609142ff60","task_key":"default--validation--6505","task_revision_id":"1","upstream_id":"6505","short_description":"HellaSwag validation 6505","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a blind or visually impaired parent [title] Child-proof your home. [step] To ensure your children are safe at home, make sure you child-proof the space. Use outlet covers and cupboard locks.\",\"endings\":\"['If your child is under 14, do not provide cleaning products that are unsafe for blind or visually impaired children. Use electrical outlets in the bathroom, as well as the kitchen, so your child cannot walk over any unsupervised surfaces.', 'Consider installing electrical outlet protectors or hiring a home electrician or contractor. [substeps] Ensure to purchase tiling and floor space.', 'Put up stair gates to ensure your child is not at risk of falling down the stairs. Keep household cleansers out of reach and small objects off the floor so your child cannot get at them.', \\\"If you have multiple children, be sure to have a' tiki' that is suitable for a single person. Assess which rule applies to the type of space space you have and establish a' no interruptions' policy.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}