{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be47c951-7d64-59b7-85e6-25f9a436e51b","task_key":"all--test--be47c951-7d64-59b7-85e6-25f9a436e51b","task_revision_id":"1","upstream_id":"","short_description":"A state legislature enacts a statute prohibiting the sale of contraceptive…","config":"all","split":"test","body":"{\"choices\":\"['to have standing.', 'to have standing jus tertii.', 'not to have standing.', 'not to have standing jus tertii.']\",\"question\":\"A state legislature enacts a statute prohibiting the sale of contraceptive devices to married persons. This state statute prohibits the use of contraceptive devices by married persons. A physician who practices in the state brings suit in federal court challenging the constitutionality of the state contraceptive statute. The physician attacks the validity of the statute on the grounds that it prevents him from giving professional advice concerning the use of contraceptives to three patients, all of whom are married, whose condition of health might be endangered by child bearing. The plaintiff is likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}