{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c2bd217-7554-55ef-8aef-e7f6ed8e3884","task_key":"default--test--4700","task_revision_id":"1","upstream_id":"4700","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The development of direct trade links between Western Europe and India', 'The colonization of North and South America by Western Europeans', 'The establishment of the Silk Road', 'The increasing development of seafaring technologies', 'The beginning of the Industrial Revolution', 'The scientific revolution', 'The development of the printing press', 'The emergence of the Ottoman Empire', 'The discovery of Australia by Western Europeans']\",\"question\":\"This question refers to the following information.\\n\\\"At least one of the [world's] societies would have to somehow enormously increase its productivity [in order to achieve global hegemony]. That quantum jump would have to be made before the various scientific, technological, agricultural, and industrial revolutions on which our post-quantum-leap world rests. It could only be accomplished by exploiting the ecosystems, mineral resources, and human assets of whole continents outside the lands of the society making the jump. Western Europe did just that by means of its brutality and guns and, more important, by geographical and ecological luck.\\\"\\nCopyright © 2015 Cambridge University Press.\\nAlfred Crosby, historian, Ecological Imperialism, 2004\\nCrosby's argument in the passage is most likely a response to which of the following developments of the period 1450–1750 C.E.?\",\"src\":\"ori_mmlu-high_school_world_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}