{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e4813f1-c911-5aff-a15c-d32db3c98e97","task_key":"default--validation--23912","task_revision_id":"1","upstream_id":"23912","short_description":"HellaSwag validation 23912","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to treat a burn using honey [title] Recognize a first-degree burn. [step] Burns can happen pretty easily. You can get burned by heat, fire, the sun, electricity, boiling liquids like water, sauces and other foods, and chemicals.\",\"endings\":\"['They might also seep through the skin or spread or break through the tissue. [substeps] First-degree burns can occur in small amounts, usually only rarely.', 'The most severe burns (again, the ones with high water levels) need to be treated in more extreme ways. Always wash your hands before you apply honey.', 'A first-degree burn is the least severe, affecting the topmost layers of your skin. [substeps] A first-degree burn will be red and painful.', \\\"This usually begins when you scrape your skin, leaves cracks or cuts on your skin, and spreads to other parts of your body. After you've been exposed to a new burn, you'll notice that it has increased in intensity.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}