{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"125304e5-6848-5bd5-a0f3-b2cc286b9caa","task_key":"default--validation--32931","task_revision_id":"1","upstream_id":"32931","short_description":"HellaSwag validation 32931","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to decorate a grave site [title] Pick the best floral option for the time of year. [step] When fresh flowers are available, especially in the spring and summer, most cemeteries would prefer that you bring fresh flowers. However, if you want something else, consider a small flowering shrub, which will continue to bring beauty to the grave site for many years.\",\"endings\":\"['You might also try autumn humps, which can be enjoyed with bare feet. [title] Let the flower grow directly from the soil.', '[substeps] If you want a flowering shrub or flowers, pick something small, native, and easy to maintain. Check with a local nursery to ask what would be best for your area.', \\\"[substeps] It's also an ideal time to plant flowers like roses or daisies. If you prefer lower temperatures, consider spinach or brunei.\\\", '[substeps] You may want to choose other flowers to cover the inside of the grave site, such as lilies, lettuce, or grasses. If you want something more decorative, consider various flower themes that would be favorable for the surface of the grave site.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}