{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f1bc9751-dbae-51d2-840a-63ecff583e22","task_key":"default--test--4990","task_revision_id":"1","upstream_id":"4990","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The emancipation terms relegated former serfs to communal farming, thus keeping them in perpetual poverty, and made wealthy conservatives nervous.', 'The emancipation led to a rise in crime rates as former serfs struggled to adapt to their newfound freedom and economic instability.', 'The emancipation caused a decline in the agricultural sector as former serfs abandoned farming in favor of other professions.', \\\"The program was a complete disaster as agricultural output plummeted until Pyotr Stolypin's reforms made privatized Russian farming profitable.\\\", 'No longer tied to the land, former serfs moved to the cities, thus spurning the commencement of an industrial revolution in Russia.', 'The emancipation led to a redistribution of land and wealth, leading to the creation of a middle class in Russia.', 'The emancipation led to a significant decrease in the population of serfs as they were given the freedom to migrate to other countries.', 'The emancipation led to widespread rebellion among the serfs, leading to civil unrest and political instability.', 'The emancipation resulted in an immediate improvement in the living conditions of the serfs, leading to increased productivity and economic growth.', 'Food production soared, which allowed Russians to export agricultural goods while investing the profits into their industrial sector.']\",\"question\":\"This question refers to the following information.\\nThe text below is the government proclamation.\\nOn the basis of the above-mentioned new arrangements, the serfs will receive in time the full rights of free rural inhabitants.\\nThe nobles, while retaining their property rights to all the lands belonging to them, grant the peasants perpetual use of their household plots in return for a specified obligation[; . . . the nobles] grant them a portion of arable land fixed by the said arrangements as well as other property. . . . While enjoying these land allotments, the peasants are obliged, in return, to fulfill obligations to the noblemen fixed by the same arrangements. In this status, which is temporary, the peasants are temporarily bound. . . .\\n[T]hey are granted the right to purchase their household plots, and, with the consent of the nobles, they may acquire in full ownership the arable lands and other properties which are allotted them for permanent use. Following such acquisition of full ownership of land, the peasants will be freed from their obligations to the nobles for the land thus purchased and will become free peasant landowners.\\nWE have deemed it advisable:\\n3. To organize Peace Offices on the estates of the nobles, leaving the village communes as they are, and to open cantonal offices in the large villages and unite small village communes.\\n4. To formulate, verify, and confirm in each village commune or estate a charter which will specify, on the basis of local conditions, the amount of land allotted to the peasants for permanent use, and the scope of their obligations to the nobleman for the land.\\n6. Until that time, peasants and household serfs must be obedient towards their nobles, and scrupulously fulfill their former obligations.\\n7. The nobles will continue to keep order on their estates, with the right of jurisdiction and of police, until the organization of cantons and of cantonal courts.\\n—Alexander II, \\\"The Abolition of Serfdom in Russia,\\\" Manifesto of February 19, 1861\\nWhich of the following best describes the long-term effects of Tsar Alexander II's emancipation?\",\"src\":\"ori_mmlu-high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}