{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eec2ef28-8d0e-59d9-a78f-e9c748ce730f","task_key":"default--validation--12285","task_revision_id":"1","upstream_id":"12285","short_description":"HellaSwag validation 12285","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to make your eyes stop hurting [title] Flush your eyes out with eyewash. [step] If you have not already done so, rinse your eyes out with a commercial eyewash or water if that's what you have on hand. F the problem is as a result of a contaminant such as a piece of dirt, this may be enough to solve the problem.\",\"endings\":\"['Make sure the water and/or solution is at a temperature of between 60 ° f (15. 6 ° c) and 100 ° f.', '[substeps] Coffee, sports drinks, grocery products, and beverages containing chemicals cause blemishes. Sodium hydroxide is thought of as a known food allergen and can irritate tonsils.', \\\"If the eyewash is not what you desire, consider lining your eyes with mineral oil or eyedrops. Don't swallow these oils or eyedrops while in the shower, as they can further irritate your eyes.\\\", '[substeps] You could purchase a regular eyewash from a drug store. Rinse thoroughly and swish the water around your eyes to remove any dead microbes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}