{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b6288b2-2841-55a4-ad0a-3dc47dc2c7a5","task_key":"all--test--1b6288b2-2841-55a4-ad0a-3dc47dc2c7a5","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Intervention by the United States and European powers in the independence of Latin American countries', 'Nationalism in Latin America pressing for stronger unity between the former Spanish colonies of South America', 'Slave rebellions fragmenting newly independent Latin American states', 'Sectionalism of elites in the various territories of newly independent Latin American states']\",\"question\":\"This question refers to the following information.\\nSource 1:\\n\\\"The Sovereign Congress of Venezuela, to which authority the people of the Republic of the New Granada have voluntarily stood by.\\nConsidering:\\n1. \\\"That united in a single Republic, the provinces of Venezuela and the New Granada have all proportions and ways to elevate themselves to a higher grade of power and prosperity.\\n2. \\\"That constituted in separate republics, for any more stronger the ties that these have united them, so far from taking advantages of so many advantages, they would hardly consolidate and make respect their sovereignty.\\n3. \\\"That these truths, highly penetrated by superior talented men and of an enlightened patriotism, had moved the governments of both republics to convene in a reunion that the vicissitudes of wars decreed and decree the following fundamental Law of the Republic of Colombia:\\nARTICLE 1. The Republics of Venezuela and New Granada are from this day on united in a single one under the glorious title of Republic of Colombia….\\nARTICLE 4. The Executive Power of the Republic will be vested on the President and in case of his defect a Vice President and his replacement will be appointed interimly by the acting Congress.\\\"\\nFundamental Law Establishing Gran Colombia, passed by the Congress of Angostura convened by Simon Bolivar, 1819\\nSource 2:\\n\\\"IN THE NAME OF THE HOLY TRINITY.\\nTITLE 1 - Of the Empire of Brazil, its Territories, Government, dynasty, and Religion.\\nArt. 1. The EMPIRE of Brazil is the political association for all Brazilian citizens. These will form a free and independent nation, and will not form any other union or federation, which may cause it to lose its own Independence.\\nArt. 2. Its territory is divided into provinces in the form in which it currently finds itself; these may be subdivided, as is required for the good of the state.\\nArt. 3. Its government is a Hereditary Monarchy, constitutional, and Representative.\\nArt. 4. The prevailing dynasty is that of the Lord Dom Pedro I, the sitting Emperor and Perpetual Defender of Brazil.\\nArt. 5. The Catholic Apostolic Roman Religion will continue to be the religion of the Empire. Followers of all other religions are allowed to worship within their households, or particular houses for that intent, so long as there is nothing indicating this on the exterior of the Temple.\\\"\\nConstitution of the Empire of Brazil, March 25, 1824\\nWhich of the following would most undermine the expectations expressed in Article 1 of Source 1?\",\"subject\":\"high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}