{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e7b0a87-89ba-561c-9d0f-75e8122a9764","task_key":"all--test--2e7b0a87-89ba-561c-9d0f-75e8122a9764","task_revision_id":"1","upstream_id":"","short_description":"A state election code provides that any political organization whose candidate…","config":"all","split":"test","body":"{\"choices\":\"[\\\"unconstitutional, because it is a violation of the First and Fourteenth Amendments' rights of free speech.\\\", \\\"unconstitutional, because it is a violation of the Fourteenth Amendment's due process clause.\\\", \\\"unconstitutional, because it violates the Fifteenth Amendment's voting provisions.\\\", 'constitutional, because the election code is nonviolative of the equal protection clause.']\",\"question\":\"A state election code provides that any political organization whose candidate received 20% or more of the vote at the most recent gubernatorial or presidential election is a political party; other political organizations are construed as political bodies. Political parties conduct primaries, and the winning candidates in each office may have their names placed on the ballot. However, nominees of political bodies or any independent candidate may only have their names placed on the ballot if they file a nominating petition signed by not less than 7% of those eligible to vote at the last election for the offices that they are seeking. The time for circulating the petition is 180 days, which is also the deadline governing party candidates in party primaries. An independent candidate, who desires to be listed on the ballot for the office of governor, challenges the constitutionality of this election law. The court will most likely declare this statute\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}