{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfca9276-2ba7-54f4-886b-f5fa2de17564","task_key":"default--validation--39436","task_revision_id":"1","upstream_id":"39436","short_description":"HellaSwag validation 39436","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to calm a baby [title] Determine if the baby needs a diaper change. [step] Many babies get upset when they are in need of a diaper change. The feeling of the dirty or wet diaper against their skin is uncomfortable.\",\"endings\":\"['To help determine if the baby needs to change, examine the aftercare package containing the diapers. The diaper change kit will contain instructions regarding how much diaper change to put on the baby and how much to put on the baby.', 'The baby may feel chilly, which can lead to irritation or nausea. The child may also experience abdominal pain, especially in their stomach.', \\\"Waiting too long to change a baby's diaper can lead to diaper rash, which is very uncomfortable. [substeps] If you are using disposable diapers, the diaper will begin to sag or appear full.\\\", 'If the baby is crying or you feel like you need to use the bathroom, it might be time to change the diaper. Keep a diaper book with you and look it up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}