{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d65e3931-a265-5863-8ff2-47cc1b551763","task_key":"default--validation--36276","task_revision_id":"1","upstream_id":"36276","short_description":"HellaSwag validation 36276","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to choose garden lighting [title] Pick stake lights for pathways. [step] These lights are brighter than ambient lights, so they're meant to go along pathways. They're also easy to install because they have stakes built in, so you can just push or hammer them into the ground as needed.\",\"endings\":\"[\\\"Make sure you know exactly how many stakes you're using-how many people will be using the assigned place at one time. [substeps] Simple stakes aren't ideal to install, though.\\\", \\\"They're also usually hard to fit directly into doorways, so they tend to be more expensive. [substeps] Stake lights are thought to be helpful for a wide variety of paths.\\\", \\\"[substeps] Lighted paths make it easier to get around outside at night. Solar-powered lights work well to illuminate pathways because you don't need a cord that could trip people up.\\\", \\\"However, they still need to be decorative. Most vines in gardens can live on one stake set, and these lights aren't necessary unless the vine was larger than one stake in the ground.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}