{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9003d2c4-8237-579d-a665-6bfa4c96961e","task_key":"default--validation--8508","task_revision_id":"1","upstream_id":"8508","short_description":"HellaSwag validation 8508","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to remove wet wax from infected ears [title] See a doctor to inspect your ears. [step] If at all possible, have a doctor examine your ears and remove any material from them, rather than doing it yourself. [substeps] Doctors are experts, and will be able to diagnose the problem accurately.\",\"endings\":\"['You should make an appointment, and speak up promptly. You can always say, \\\" by all means see a doctor.', 'However, if you notice any changes in your ears or wax, such as shrinking and peeling, it might be a sign of an infection or health problem. Always make it a habit to see a doctor before making a complete diagnosis for an ear that is infected.', 'It is difficult to see inside your ear on your own. You can easily damage your inner ears if you use improper tools or techniques to clean them.', \\\"While wax is harmless, it's a necessary risk factor. [title] Purchase a pair of non prescription hearing protection drops.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}