{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"55cbb049-2349-5a45-a2cf-fcd87c5a473c","task_key":"all--test--55cbb049-2349-5a45-a2cf-fcd87c5a473c","task_revision_id":"1","upstream_id":"","short_description":"A couple were married, and four months later, they wished to obtain a divorce.…","config":"all","split":"test","body":"{\"choices\":\"['constitutional, because the requirement promotes a compelling state interest.', 'constitutional, because it is within the area of state action.', 'unconstitutional, because it constitutes a violation of the equal protection clause.', 'unconstitutional, because it constitutes a violation of the privileges and immunities clause of Article IV, Section 2.']\",\"question\":\"A couple were married, and four months later, they wished to obtain a divorce. However, the state in which they were married prohibits a couple from obtaining a divorce until they have resided in the state for at least 12 months. In an action by the couple challenging the constitutionality of the residency requirement of the state divorce law, the court will most likely declare the provision\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}