{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dcf9bd8-a0ce-5425-a879-3dce8e309dfe","task_key":"default--validation--36145","task_revision_id":"1","upstream_id":"36145","short_description":"HellaSwag validation 36145","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to play with your newborn [title] Make faces at each other. [step] In terms of engagement and enjoyment, there's nothing more enjoyable for your newborn than a good face-making session. In fact, newborns even prefer faces to toys.\",\"endings\":\"['Words and expressions help children remember that face-to-face communication is just that, a game of word association. [substeps] Make your face visible to your newborn by raising one hand slightly-or both-and then raising it both in front of you.', \\\"Once you've bred your baby, face-making is increasingly becoming an all-time fun activity. [substeps] This is especially true with older babies, who are typically not uninterested in your faces-they can still contemplate and cry with you if they see you smiling and laughing.\\\", 'Children, even those who are younger and who are used to connecting to their caregiving partner, can benefit from this experience. [substeps] For example, start by bumping their faces lightly with both hands.', 'Further, your baby will be especially excited to observe wide-open eyes and genuine smiles. Look your baby in the eye and respond to any actions the baby makes by changing your expression.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}