{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8581844-9829-58a3-9d6e-5ad9a4c9abf9","task_key":"all--test--d8581844-9829-58a3-9d6e-5ad9a4c9abf9","task_revision_id":"1","upstream_id":"","short_description":"A police officer entered a private home in response to a reported disturbance.…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The statute violates the men's right to liberty encompassed within the constitutional guarantee of substantive due process.\\\", \\\"The federal constitution cannot be used to interfere with a state's right to regulate sexual morality.\\\", \\\"The statute is unconstitutional primarily because it infringes the men's right to freedom of expression under the First Amendment.\\\", \\\"The relationship in question is not protected by the right to privacy and is subject to a state's criminal regulation.\\\"]\",\"question\":\"A police officer entered a private home in response to a reported disturbance. She found no disturbance but did witness the two male residents engaging in consensual sexual relations with each other. She arrested and charged them with \\\"deviate sexual intercourse\\\", a state law that prohibits two persons of the same sex from engaging in mutual sexual relations. After conviction and unsuccessful appeals in the state courts, the men appealed to the United States Supreme Court. What is the likely holding and reasoning of the Supreme Court?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}