{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1cf649f-2a21-58a0-bbdd-d0e79f44e8a3","task_key":"default--test--4985","task_revision_id":"1","upstream_id":"4985","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['It had been expanded upon to include non-empirical forms of reasoning.', 'It had become a core principle of European culture.', 'It had stagnated to the point that the common person had begun to search for a new organizing principle of life.', 'It was only used in select areas of academic study.', 'It had been replaced entirely by a different method of reasoning.', 'It had become so widely accepted that it was no longer questioned.', 'It had weakened to the point of irrelevance.', 'It had been refined and changed by so many people that it had become unrecognizable to those such as Bacon who had pioneered it.', 'It had been altered to incorporate religious beliefs into the scientific process.', 'It had been completely dismissed by the scientific community.']\",\"question\":\"This question refers to the following information.\\n\\\"XI. As the present sciences are useless for the discovery of effects, so the present system of logic is useless for the discovery of the sciences.\\nXIX. There are and can exist but two ways of investigating and discovering truth. The one hurries on rapidly from the senses and particulars to the most general axioms, and from them, as principles and their supposed indisputable truth, derives and discovers the intermediate axioms. This is the way now in use. The other constructs its axioms from the senses and particulars, by ascending continually and gradually, till it finally arrives at the most general axioms, which is the true but unattempted way.\\nXXII. Each of these two ways begins from the senses and particulars, and ends in the greatest generalities&…\\nXXXVI. We have but one simple method of delivering our sentiments, namely, we must bring men to particulars and their regular series and order, and they must for a while renounce their notions, and begin to form an acquaintance with things.\\\"\\nFrancis Bacon, English philosopher and essayist, Novum Organum, 1620\\nBy the 1800s, the method of empirical reasoning reflected in the passage had undergone which of the following changes?\",\"src\":\"ori_mmlu-high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}