{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4aaf98fa-7bea-5a9a-9901-b9dccff772fc","task_key":"all--test--4aaf98fa-7bea-5a9a-9901-b9dccff772fc","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"[\\\"After the drafting of the Constitution, continued debates about states' rights and competing economic interests led to the formation of political parties, such as the Federalists and the Democratic-Republicans.\\\", 'Although Washington warned against the influence of political parties, he reluctantly accepted the endorsement of the Federalist party.', 'Political parties did not exist until the War of 1812, when Federalists asserted control over foreign policies with France and Britain.', 'Two major political parties dominated early presidential elections until John Quincy Adams founded the Whig party in 1824.']\",\"question\":\"This question refers to the following information.\\n\\\"Some men look at constitutions with sanctimonious reverence, and deem them like the ark of the covenant, too sacred to be touched. They ascribe to the men of the preceding age a wisdom more than human, and suppose what they did to be beyond amendment….But I know also, that laws and institutions must go hand in hand with the progress of the human mind. As that becomes more developed, more enlightened, as new discoveries are made, new truths disclosed, and manners and opinions change with the change of circumstances, institutions must advance also, and keep pace with the times.\\\"\\nThomas Jefferson, 1816\\nWhich of the following statements is most accurate regarding the existence of political parties in the early United States?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}