{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41bbdf26-8f52-590d-bf02-06fbc1696b36","task_key":"default--test--5941","task_revision_id":"1","upstream_id":"5941","short_description":"Which statement best describes the relationship between realist international…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Peace researchers and realists were in full agreement on the focus of their studies, both advocating for a reduction in nuclear arms as the best path to peace.', 'Although peace researchers were critical of the realist focus on military strength, both groups agreed that the Soviet bloc posed a significant threat to Western liberal democracies.', 'Realists and peace researchers shared the same views and advocated for disarmament and peaceful coexistence as the best way to resolve the East-West divide of the Cold War.', 'Whilst peace researchers indentified the escalation of the arms race as a result of failures by the League of Nations, realist scholars deemed it to be the natural progression of ideological differences between the Soviet bloc and Western liberal democracies. This resulted in irreconcilable differences between the study agendas of realists and peace researchers.', 'The East-West divide of the Cold War gave rise to a sharp divide between realists and early peace studies theorists. Whilst realists wished to focus research on that which would favour the survival of Western liberal democracies, advocates of peace studies regarded this approach as being narrow and Western ethnocentric.', 'Both peace researchers and realists advocated an academic focus on the escalation of the arms race. Although tensions existed between the groups in terms of their proposed solutions to this crisis, there existed a consensus that the Soviet bloc should be eliminated as a nuclear threat, thus favouring the survival of Western liberal democracies.', 'Realists and peace researchers had a common goal of promoting peace, but differed significantly in their approaches, with the former advocating for military supremacy and the latter for diplomacy and negotiation.', 'The realist school and peace researchers had minor disagreements on their study agendas, but both believed in the importance of maintaining a balance of power to achieve security.']\",\"question\":\"Which statement best describes the relationship between realist international relations theorists and peace researchers in the early 1960s?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}