{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cfe112aa-33be-580c-8069-8ed37ecd39d1","task_key":"default--validation--5867","task_revision_id":"1","upstream_id":"5867","short_description":"HellaSwag validation 5867","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to get stuff out of your eye [title] Wash your hands. [step] Even if your hands don't seem dirty, it's important to wash them if you're going to touch your eye. You don't want to remove an object from your eye only to infect it with something worse.\",\"endings\":\"[\\\"[substeps] After washing your hands with a gentle soap, use warm, soapy water to clean your eye. If you'd rather not use hand sanitizer, wash your hands with warm, soapy water.\\\", \\\"Don't touch your eye with a hairbrush or similar item. [substeps] If you suspect eye germs, act quickly and remove them before your client comes to visit.\\\", \\\"If possible, use a mild soap or antibacterial hand soap. [substeps] If you're in a public place, however, to catch and remove a substance, go ahead.\\\", \\\"[substeps] Wash your hands thoroughly with soap and clean water for at least 20 seconds. This ensures that you don't get bacteria or other contaminants in your eyes.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}