{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6faadcc9-b1af-5a46-b905-22da09512ff1","task_key":"default--validation--3301","task_revision_id":"1","upstream_id":"3301","short_description":"HellaSwag validation 3301","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy baby shoes [title] Buy shoes when your baby starts taking steps. [step] Infants don't need shoes, because you're always carrying them everywhere. But once your baby officially starts learning to walk, it's a good idea to have at least one pair of shoes on hand to provide protection from dirt, germs, and dangerous objects.\",\"endings\":\"[\\\"[title] Take steps often, but don't walk in shoes for long periods of time. [step] Babies develop foot arthritis when they are walking, so you can't expect your baby to notice until they get comfortable wearing shoes.\\\", \\\"[substeps] It's fine to let your baby practice walking at home without shoes. It might even be easier for them to figure out how to balance out those first wobbly steps when they are barefoot.\\\", \\\"[substeps] A child needs shoes around three months old so that they're not worn out in the yard. A toddler needs shoes at six months old to learn to walk.\\\", '[substeps] If your baby is unsure about how to walk, refer to a shoe shop, or ask someone in your department if your child can borrow shoes. Consider purchasing shoes from a store that has a treadmill or other machine that your baby may like.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}