{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c235734-683a-50aa-9b70-baa2cc9fb986","task_key":"default--validation--40565--variant-d18a2a992626","task_revision_id":"1","upstream_id":"40565--variant-d18a2a992626","short_description":"HellaSwag validation 40565","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to remove contact lenses [title] Wash your hands. [step] Harmful pathogens can enter the eye via the lenses and cause eye infection or conjunctivitis. Use antibacterial soap and warm water to wash your hands.\",\"endings\":\"['Dry your hands thoroughly with a clean towel. [substeps] Keeping good hand hygiene regimen not only protects your contacts from harmful pathogens, but also your eyes in general.', 'Make sure you clean them well so you can safely remove contact lenses. [substeps] Pour some baking soda into a bowl or container.', '[substeps] Lather your hands with warm water for five to ten seconds. Drain your hand water into the sink and use soap if it is not clogged.', '[substeps] Before washing a contact lens, wipe your lenses with a paper towel to remove dirt or soap residue. Wash your hands for at least 20 seconds before putting them back on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}