{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d1c21fc-fbe7-5225-9bb5-358ce6fbdb7f","task_key":"default--test--4955","task_revision_id":"1","upstream_id":"4955","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The ideals of communism as stated by Joseph Stalin', 'The ideals of democratic socialism as stated by Salvador Allende', 'The ideals of social democracy as stated by Bernie Sanders', 'The ideals of globalization as evidenced by multinational trade blocs such as NAFTA or the European Union', 'The ideals of religious fundamentalism as stated by the Ayatollah Khomeini', 'The ideals of capitalist theory as stated by Milton Friedman', 'The ideals of classical liberalism as stated by Adam Smith', 'The ideals of nationalism as evidenced by the rise of populist movements in the 21st century', 'The ideals of the Enlightenment as stated by political revolutionaries such as Maximilian Robespierre', 'The ideals of fascism as stated by Benito Mussolini']\",\"question\":\"This question refers to the following information.\\n\\\"The spontaneous forces of capitalism have been steadily growing in the countryside in recent years, with new rich peasants springing up everywhere and many well-to-do middle peasants striving to become rich peasants. On the other hand, many poor peasants are still living in poverty for lack of sufficient means of production, with some in debt and others selling or renting out their land. If this tendency goes unchecked, the polarization in the countryside will inevitably be aggravated day by day. Those peasants who lose their land and those who remain in poverty will complain that we are doing nothing to save them from ruin or to help them overcome their difficulties. Nor will the well-to-do middle peasants who are heading in the capitalist direction be pleased with us, for we shall never be able to satisfy their demands unless we intend to take the capitalist road. Can the worker-peasant alliance continue to stand in these circumstances? Obviously not! There is no solution to this problem except on a new basis. And that means to bring about, step by step, the socialist transformation of the whole of agriculture simultaneously with the gradual realization of socialist industrialization and the socialist transformation of handicrafts and capitalist industry and commerce; in other words, it means to carry out co-operation and eliminate the rich-peasant economy and the individual economy in the countryside so that all the rural people will become increasingly well off together. We maintain that this is the only way to consolidate the worker-peasant alliance.\\\"\\nMao Zedong, On the Question of Agricultural Co-operation, 1955\\nMao's view of the cooperation of peasant labor most directly reflects the influence of which of the following?\",\"src\":\"ori_mmlu-high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}