{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43c3491f-dbaa-5dd8-89a2-cbfa35f8b69f","task_key":"default--validation--37637","task_revision_id":"1","upstream_id":"37637","short_description":"HellaSwag validation 37637","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to garden when you have a dog [title] Plant edible flowers such as butterfly iris and roses. [step] In a perfect world, your dog wouldn't touch, let alone consume, any of your garden plants. In the event that they do experiment with some of your backyard flora, though, you want to make sure that this accidental consumption is non-toxic.\",\"endings\":\"['Planting flowers and plants in a cage encourages aggressive growth, so many plants develop less aggressive behavior than plants that clog your home. [substeps] Keep the garden on top of soil.', '[title] Separate the flowers and drift them into separate containers. [step] If your garden is small, heavy objects can weigh them.', 'Opt for flowers that are wholly and safely edible for peace of mind. [substeps] In contrast, avoid flowers such as foxgloves, rhododendron, azaleas, lily of the valley, and everything from the buttercup family.', \\\"Plant your unique garden tomato plant with edible flowers such as violet, red rose, and plums. If your dog won't eat the plant, that can always go a long way towards a healthy dog diet.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}