{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d538c832-fd00-583c-9df3-7ff4f970bb05","task_key":"default--validation--17708","task_revision_id":"1","upstream_id":"17708","short_description":"HellaSwag validation 17708","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to give a baby a bath [title] Wear appropriate clothing. [step] Roll up your long sleeves, remove your jewelry, and take off any other items, such as watches, that may get in your way. Know that bathing a baby can be a wet operation and be prepared to find a change of clothes afterward.\",\"endings\":\"['If you are bathing a baby, then it is best to use a diaper hose or baby bubble bath. If you are bathing a baby while breastfeeding, use a diaper hose or diaper bubble bath.', \\\"You want to wear something you don't care about so you can properly bathe your baby. [title] Get all of your supplies in order.\\\", \\\"[substeps] Make sure that you water both hands until your hands are completely dry before applying any gel products and soaps. [title] Avoid applying creams, oils, or serums to your baby's skin.\\\", \\\"[substeps] Using hot water can impair the baby's circulation, leading to injuries or illness. In addition, the baby may be nervous, especially since she will be moving, and may not be able to properly bathe herself.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}