{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d199e183-2a9a-52c5-9fc7-29c8696111d0","task_key":"default--validation--2896","task_revision_id":"1","upstream_id":"2896","short_description":"HellaSwag validation 2896","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to keep hornets away [title] Cover all food sources, such as compost piles and trashcans. [step] Unlike bees, wasps and hornets like to eat meat and other protein foods, as well as sweet nectars. Hornets will remember the food source and continue to come around after it has been sealed off or taken away, so it's best to prevent all open-air garbage.\",\"endings\":\"['[substeps] The first thing wasps most likely to notice and become interested in is beans and fruits, both natural and edible. Female hornets recognize insects and are attracted to them on a daily basis.', '[title] Reduce sweet smells during summer or fall. [step] This is the point when wasps prefer blossoms, juices and fruit.', '[substeps] Wasps and hornets also like to eat poultry, fruits, vegetables, and natural foods like carrots, lettuce, and oranges. Keep them away from anything that has bees on it.', '[substeps] Spread cover around plants and garden beds that are organic. Only put out as much as you can-these plants are bait for wasps.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}