{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17f8e7a5-fd19-5fff-a5b9-2c9cd51bb3c3","task_key":"default--test--4699","task_revision_id":"1","upstream_id":"4699","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The implementation of joint-stock companies', 'The exploitation of labor in colonized regions', 'The exploration of new sea routes to Asia and the Americas', 'The establishment of trading posts along Asian and African coasts', 'The exchange of food sources between the Americas and Europe', 'The spread of Christianity to non-European regions', 'The development and application of steam-powered technologies', 'The discovery and colonization of the Americas', 'The development of advanced naval technology', 'Refinement of gunpowder technologies']\",\"question\":\"This question refers to the following information.\\n\\\"At least one of the [world's] societies would have to somehow enormously increase its productivity [in order to achieve global hegemony]. That quantum jump would have to be made before the various scientific, technological, agricultural, and industrial revolutions on which our post-quantum-leap world rests. It could only be accomplished by exploiting the ecosystems, mineral resources, and human assets of whole continents outside the lands of the society making the jump. Western Europe did just that by means of its brutality and guns and, more important, by geographical and ecological luck.\\\"\\nCopyright © 2015 Cambridge University Press.\\nAlfred Crosby, historian, Ecological Imperialism, 2004\\nBased on your knowledge of world history, which of the following contributed LEAST to Western European global hegemony between 1450 and 1750 C.E.?\",\"src\":\"ori_mmlu-high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}