{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00f2b28e-4b97-51c6-8269-256c95f908e6","task_key":"default--validation--11450","task_revision_id":"1","upstream_id":"11450","short_description":"HellaSwag validation 11450","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to distinguish which brand of disposable diaper a baby is wearing [title] Walk into the child's baby nursery or bedroom. [step] Sometimes the package of diapers may be stashed in this room. Sometimes the packages can be found under the crib or bed.\",\"endings\":\"['Or can sometimes even be stashed in a corner; the possibilities are endless, so look hard. [substeps] If you are 100% right that the package is impossible to find in the nursery, ask your friend where she keeps the bag of new disposable diapers.', \\\"If they're not accessible by the baby, make note of where they are and why they have this particular brand. Note if the place is stocked with children's clothes, bedding, blankets, and so on.\\\", \\\"[substeps] Baby diapers, especially those with toggle backs, are located close to the baby's head and are meant to be changed every three to five days to prevent diaper rash (also known as psittacosis). [title] Look for either a diaper bag or a box.\\\", '[title] Use the diaper app to find the plastic wrappers. [step] Many brands such as petit baby diapers or lize were used to use as the diapers of babies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}