{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e863b3ac-bd8b-56f6-83a8-640eee921701","task_key":"default--validation--27112","task_revision_id":"1","upstream_id":"27112","short_description":"HellaSwag validation 27112","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make organic baby wipes [title] Select a roll of organic paper towels. [step] These act as the actual wipe. Experiment with different brands and textures until you find the one that works best with your baby's skin.\",\"endings\":\"[\\\"It's important to use organic paper towels as non organic paper towels may contain bleach, dye and other chemicals commonly used in the production of paper products. [substeps] As a general rule of thumb, name brand paper towels make better wipes than the cheaper, generic versions.\\\", \\\"You can use a paper towel used for diaper wipes, but you'll be able to use the same one for other wipes as well. [substeps] You can use three-or four-sided wipes to help you blend all of the milk, yogurt, and water you have mixed together and in one container.\\\", \\\"If you're doing a homemade lotion don't use paper towels; they should also be non-paper. Roll out a large roll of smaller paper towels, keeping in mind the overall odor of the pad.\\\", \\\"[substeps] Spread the dry cloth over the entire surface of your baby's skin, and avoid touching the dry cloth (trimmer, comb, or pumice). [title] Place the bottle over a bowl of warm water for 10 minutes.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}