{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b67d1ae8-48ef-59dd-b6ae-eec4621e2716","task_key":"default--validation--2955--variant-974dd28dece4","task_revision_id":"1","upstream_id":"2955--variant-974dd28dece4","short_description":"HellaSwag validation 2955","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to plant a shrub [title] Consider possible locations. [step] It's important to choose a plant that will thrive where you place it. You may not have a location that's right for a shrub you want, but you can evaluate the available space to choose a plant that's perfect for that spot.\",\"endings\":\"['Before you buy your plant, take time to evaluate what type of shrub will thrive in the location you choose. Make notes of sun exposure, temperature fluctuations, soil moisture, root growth area, and soil composition and acidity.', \\\"[substeps] The most popular choice, find a bush where it's about the size of a plant you want and tear it apart. Poke holes in the top, bottom, and sides of the bush.\\\", \\\"[substeps] You can plant your shrub anywhere you like, whether it be your backyard or outside. Forget about the weather when you plant it if it's indoors or outdoors.\\\", \\\"[substeps] You can find a standard topographical map online or at a gardening store, and you can also ask a store employee what plants grow best in a given spot. If you decide to place your shrub in an area that's ideal for outdoor use, research the location's approximate brilliance before you plant.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}