{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f58999d-45e2-5af2-a4c5-c216a780fd40","task_key":"default--test--4807","task_revision_id":"1","upstream_id":"4807","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The discovery and colonization of Australia', 'The cultural exchange with Native American tribes', 'The development of a centralized government in Europe', 'The reformation of Catholic Christianity', 'The conquest of the Aztec Empire', 'The industrial revolution in Britain', 'The isolationism of Tokugawa Japan', 'The colonization of the interior of Africa', 'The construction of the Great Wall of China', 'The establishment of trade routes with China']\",\"question\":\"This question refers to the following information.\\n\\\"At least one of the [world's] societies would have to somehow enormously increase its productivity [in order to achieve global hegemony]. That quantum jump would have to be made before the various scientific, technological, agricultural, and industrial revolutions on which our post-quantum-leap world rests. It could only be accomplished by exploiting the ecosystems, mineral resources, and human assets of whole continents outside the lands of the society making the jump. Western Europe did just that by means of its brutality and guns and, more important, by geographical and ecological luck.\\\"\\nCopyright © 2015 Cambridge University Press.\\nAlfred Crosby, historian, Ecological Imperialism, 2004\\nWhich of the following would best support the author's assertion regarding the \\\"quantum jump\\\" that would help Western Europe achieve global hegemony between 1450 and 1750 C.E.?\",\"src\":\"ori_mmlu-high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}