{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41b798da-bba6-5164-9456-5d19456352dd","task_key":"all--test--41b798da-bba6-5164-9456-5d19456352dd","task_revision_id":"1","upstream_id":"","short_description":"Congress, under intense lobbying pressure has enacted a statute prohibiting the…","config":"all","split":"test","body":"{\"choices\":\"['constitutional, because it is a regulation of interstate commerce.', 'constitutional, because it is a measure promoting the general welfare.', 'unconstitutional, because the law deprives the manufacturers of contraceptives of their property interest without just compensation.', 'unconstitutional, because it violates the right of privacy of contraceptive users.']\",\"question\":\"Congress, under intense lobbying pressure has enacted a statute prohibiting the sale of contraceptive devices to married persons. The act further prohibits the use of contraceptive devices by married persons. Congress claimed that the statute was passed because it might help deter illicit sexual relationships. The law is most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}