{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0824bde1-f8a3-5b9a-900f-ce75fcdb5cc5","task_key":"default--test--5055","task_revision_id":"1","upstream_id":"5055","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Recent defeat in the Crimean War convinced the tsar some domestic reforms were necessary.', 'The Tsar wanted to improve his popularity among the Russian people.', 'The spread of socialist ideas among the serfs was causing unrest.', 'The increasing population of serfs was becoming too difficult to manage.', 'A labor force to complete the Trans-Siberian Railroad was needed as well as military recruits.', 'The Tsar was motivated by a desire to modernize and industrialize Russia.', 'The Decembrist Revolt and its aftermath had convinced the young tsar to make reforms.', 'The Tsar believed that freeing the serfs would help Russia in its competition with Western powers.', 'The Tsar was influenced by the writings of liberal philosophers.', 'Enlightened rulers in Prussia and Austria had recently done the same, which pressured Alexander II to act.']\",\"question\":\"This question refers to the following information.\\nThe text below is the government proclamation.\\nOn the basis of the above-mentioned new arrangements, the serfs will receive in time the full rights of free rural inhabitants.\\nThe nobles, while retaining their property rights to all the lands belonging to them, grant the peasants perpetual use of their household plots in return for a specified obligation[; . . . the nobles] grant them a portion of arable land fixed by the said arrangements as well as other property. . . . While enjoying these land allotments, the peasants are obliged, in return, to fulfill obligations to the noblemen fixed by the same arrangements. In this status, which is temporary, the peasants are temporarily bound. . . .\\n[T]hey are granted the right to purchase their household plots, and, with the consent of the nobles, they may acquire in full ownership the arable lands and other properties which are allotted them for permanent use. Following such acquisition of full ownership of land, the peasants will be freed from their obligations to the nobles for the land thus purchased and will become free peasant landowners.\\nWE have deemed it advisable:\\n3. To organize Peace Offices on the estates of the nobles, leaving the village communes as they are, and to open cantonal offices in the large villages and unite small village communes.\\n4. To formulate, verify, and confirm in each village commune or estate a charter which will specify, on the basis of local conditions, the amount of land allotted to the peasants for permanent use, and the scope of their obligations to the nobleman for the land.\\n6. Until that time, peasants and household serfs must be obedient towards their nobles, and scrupulously fulfill their former obligations.\\n7. The nobles will continue to keep order on their estates, with the right of jurisdiction and of police, until the organization of cantons and of cantonal courts.\\n—Alexander II, \\\"The Abolition of Serfdom in Russia,\\\" Manifesto of February 19, 1861\\nWhich of the following was a major impetus in convincing Tsar Alexander II of the necessity of freeing the serfs?\",\"src\":\"ori_mmlu-high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}