{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6000cd6f-abb6-5a37-b2fa-5da37468f609","task_key":"all--test--6000cd6f-abb6-5a37-b2fa-5da37468f609","task_revision_id":"1","upstream_id":"","short_description":"Dr. Ramchandran found that his patients who brushed their teeth after lunch had…","config":"all","split":"test","body":"{\"choices\":\"[\\\"If at least 100 patients were studied, the writer's statement is justified.\\\", \\\"If a minimum of 500 patients were studied, the writer's statement is justified.\\\", \\\"At least 100 of the patients needed to have brushed their teeth after lunch for the writer's statement to be justified.\\\", \\\"No matter how many participants, the writer's statement is not justified.\\\"]\",\"question\":\"Dr. Ramchandran found that his patients who brushed their teeth after lunch had 1/20 the number of cavities in their teeth as those who didn't. After interviewing the dentist, a local newswriter reports that brushing teeth after lunch prevents cavities. Based on the dentist's research, which of the following statements is true?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}