{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f473bde-da4f-564d-9290-eb70aab25265","task_key":"default--validation--5424","task_revision_id":"1","upstream_id":"5424","short_description":"HellaSwag validation 5424","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to disinfect earplugs [title] Inspect your reusable earplugs, first. [step] You are checking to see if your earplugs are torn, bent, or excessively dirty. [substeps] Your earplugs can harden and become rigid if they are covered in a lot of earwax and skin oils from your ear canal.\",\"endings\":\"['If you are concerned that the earplugs do not have enough polish, consider replacing the lock caps. The lock caps on areas that get a lot of wax can become bent, or cracked.', 'This loss of flexibility will prevent you from getting a good seal when you use your earplugs. Your ear canal connects your outer ear with your eardrum.', \\\"Get a replacement pair if you plan to keep them. Tape the replacement tubes to your earplugs, (don't stuff them), clean them in/between tubes, and pop each tube out.\\\", 'When using a plastic cleaning device, ensure that it is fully dry before using it on your ear buds. Use a clean, lint-free cloth before applying your cleaning solution to keep your earplugs from sticking to your ear.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}