{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b19ca5f0-1038-5235-915f-fff4c554b821","task_key":"default--test--5020","task_revision_id":"1","upstream_id":"5020","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The success of the Populist party was due to the widespread implementation of the gold standard.', 'The Great Depression of the 1890s led to the success of the Populist party.', 'The Populist party gained success due to the rise of labor unions in the industrial North.', 'The success of the Populist party was a direct result of the prohibition movement.', 'After the Civil War, westward migration slowed, causing a long-term recession in many Western territories.', 'The expansion of the railroads in the West led to the success of the Populist party.', 'Western farmers resisted the mechanization of agriculture and resented government interference in rural affairs.', 'The growth of corporate power and banking interests inspired rural activists to lobby for political reform.', 'Western farmers and ranchers favored conservation and organized to promote the National Parks system.', 'The Populist party was successful due to the growing discontent with the Federal Reserve System.']\",\"question\":\"This question refers to the following information.\\n\\\"We have witnessed for more than a quarter of a century the struggles of the two great political parties for power and plunder, while grievous wrongs have been inflicted upon the suffering people. We charge that the controlling influences dominating both these parties have permitted the existing dreadful conditions to develop without serious effort to prevent or restrain them. Neither do they now promise us any substantial reform. They have agreed together to ignore, in the coming campaign, every issue but one. They propose to drown the outcries of a plundered people with the uproar of a sham battle over the tariff, so that capitalists, corporations, national banks, rings, trusts, watered stock, the demonetization of silver and the oppressions of the usurers may all be lost sight of. They propose to sacrifice our homes, lives, and children on the altar of mammon; to destroy the multitude in order to secure corruption funds from the millionaires.\\n\\\"Assembled on the anniversary of the birthday of the nation, and filled with the spirit of the grand general and chief who established our independence, we seek to restore the government of the Republic to the hands of 'the plain people,' with which class it originated. We assert our purposes to be identical with the purposes of the National Constitution; to form a more perfect union and establish justice, insure domestic tranquillity, provide for the common defence, promote the general welfare, and secure the blessings of liberty for ourselves and our posterity.\\\"\\nPopulist Party Platform, 1892\\nWhich of the following was most directly a cause of the success of the Populist party?\",\"src\":\"ori_mmlu-high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}