{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30d90409-2a9a-5f58-9861-8d62adf74adf","task_key":"default--validation--21368","task_revision_id":"1","upstream_id":"21368","short_description":"HellaSwag validation 21368","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to use everyday groceries in alternative ways [title] Use baking soda as toothpaste. [substeps] Use baking soda twice a month to brush your teeth. It'll help you to get rid of coffee or red wine stains.\",\"endings\":\"[\\\"Eating more than one cup of baking soda for breakfast can help you get rid of as many fake stains as possible. If you don't enjoy brushing your teeth, this is a good time to try using a toothpaste such as bentonite toothpaste or sherbert toothpaste.\\\", 'Mix two tablespoons of baking soda with a tablespoon of warm water in a bucket until you get a creamy, thick paste. Wet your hands and apply the paste, or put a little baking soda on a cotton pad and rub over your lips, then rinse.', '[title] Put one heaping tablespoon of baking soda into a blender. [title] Measure out one tablespoon of some dog-food mix and the juice from three lemons (or you can use packets).', 'Baking soda can react with acids and bases. This makes baking soda perfect for neutralizing acids produced by bacteria in your mouth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}