{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"187a9f05-93a1-50b1-a500-23b677c5c5bf","task_key":"default--test--5016","task_revision_id":"1","upstream_id":"5016","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['The cultural movement of the Renaissance.', 'The Enlightenment philosophy of natural rights.', 'The social contract theory.', 'The political philosophy of absolutism.', 'The economic policy of mercantilism.', 'The economic policy of capitalism.', 'The religious philosophy of conversion.', 'The religious philosophy of predestination.', 'The theory of divine right of kings.', 'The philosophy of colonialism.']\",\"question\":\"This question refers to the following information.\\n\\\"I marvel not a little, right worshipful, that since the first discovery of America (which is now full four score and ten years), after so great conquests and plantings of the Spaniards and Portuguese there, that we of England could never have the grace to set fast footing in such fertile and temperate places as are left as yet unpossessed of them. But . . . I conceive great hope that the time approacheth and now is that we of England may share and part stakes [divide the prize] (if we will ourselves) both with the Spaniard and the Portuguese in part of America and other regions as yet undiscovered.\\n\\\"And surely if there were in us that desire to advance the honor of our country which ought to be in every good man, we would not all this while have [neglected] the possessing of these lands which of equity and right appertain unto us, as by the discourses that follow shall appear most plainly.\\\"\\n—Richard Hakluyt, Divers Voyages Touching the Discovery of America and the Islands Adjacent, 1582\\nThe ideas expressed in the passage above most closely reflect the influence of which of the following?\",\"src\":\"ori_mmlu-high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}