{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"52b8f468-e597-5817-9e0a-319564cac367","task_key":"default--validation--3605","task_revision_id":"1","upstream_id":"3605","short_description":"HellaSwag validation 3605","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to control weeds [title] Refrain from disturbing the soil. [step] Heavy digging and cultivation can cause weed seeds to come to the surface and begin growing. Only disturb the soil when you have to, so that you don't inadvertently raise weed seeds.\",\"endings\":\"['[title] Remove weeds in a well-ventilated garden space. [step] If you will be supervising garden edging, you will want to be in a well-ventilated location, and away from trees or buildings.', \\\"[substeps] Allow clean-up time to occur in the midst of the weeds' harvests. Dig up the roots, and spread them around the garden bed (usually about 1-2 feet deep) to promote leaf-dusts, as well as prevent the trees from turning.\\\", '[substeps] Weed seeds can lay dormant under the soil for years. Hoeing or tilling in weed prone areas may inadvertently bring up dormant weed seeds in the process.', '[substeps] Ideally, you should be able to plant your straw garden in spring or summer. Try to keep the weeds contained to the top of the plants so that the surface is less saturated with soil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}