{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d76ef162-16c4-5796-bc93-f35944c3b881","task_key":"default--test--1723","task_revision_id":"1","upstream_id":"1723","short_description":"The state registrar of elections refused to put a candidate's name on the ballot…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The suit should have been brought as an original action in the U. S. Supreme Court.', 'The federal court cannot intervene because it would disrupt the separation of powers.', 'Under Article III of the Constitution, the federal courts are not empowered to render advisory opinions.', 'The candidate lacks standing to sue because she was never officially a member of the House of Representatives.', 'The case should have been filed in the state court where the original residency determination was made.', 'The federal court lacks jurisdiction because the House of Representatives has exclusive authority to determine the qualifications of its members.', 'The case is moot because the term of office for which the candidate was elected has already ended.', 'There is no case or controversy between the candidate and the officers of the House of Representatives.', \\\"The candidate's claims are barred by the Eleventh Amendment.\\\", 'The case presents a nonjusticiable political question.']\",\"question\":\"The state registrar of elections refused to put a candidate's name on the ballot because the registrar believed that the candidate was not a resident of the state. The registrar contended that Article I, Section 2 of the U. S. Constitution specifically required a candidate for the House of Representatives \\\"be an Inhabitant of that State in which he shall be chosen. \\\" As a consequence, the candidate filed suit in state court against the registrar of elections seeking to have her name placed on the ballot. The state court ruled in her favor and determined that the candidate did, in fact, qualify as a resident of the state. The registrar appealed the decision to the state supreme court which, in turn, affirmed the lower court ruling. Thereafter, the candidate's name was placed on the ballot. After a hotly contested election, the candidate narrowly defeated the incumbent. However, at the time that the candidate was to be seated as a House member, the House of Representatives held hearings on her qualifications and eligibility. By a two-thirds vote, the House determined that the candidate was not a resident of the state at the time of her election and refused to seat her. The candidate then brought suit in federal district court against the Speaker and other officers of the House, seeking back pay and an order that she be seated in the House of Representatives. The defendants demurred, claiming lack of subject matter jurisdiction. Which of the following is the strongest constitutional argument supporting the defendants' demurrer?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}