{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfde24d3-4240-5ebd-8e8f-fe6533c76525","task_key":"default--test--4991","task_revision_id":"1","upstream_id":"4991","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Shifts in political power due to the French Revolution', 'The rise of capitalism and market economies', \\\"New concepts of a deterministic mechanical universe based upon the discovery of Newton's mathematical laws\\\", 'Increased interfaith dialogue and cooperation', 'The rejection of religious dogma and the rise of atheism in Europe', 'Advances in the scientific method and the pursuit of empirical evidence', 'Europeans gaining a more thorough understanding of the rich cultural diversity through trade and travel']\",\"question\":\"This question refers to the following information.\\nThe following excerpt is from a pamphlet.\\nYou will do me the justice to remember, that I have always strenuously supported the Right of every man to his own opinion, however different that opinion might be to mine. He who denies to another this right, makes a slave of himself to his present opinion, because he precludes himself the right of changing it.\\nThe most formidable weapon against errors of every kind is Reason. I have never used any other, and I trust I never shall.\\nThe circumstance that has now taken place in France of the total abolition of the whole national order of priesthood, and of everything appertaining to compulsive systems of religion, and compulsive articles of faith, has not only precipitated my intention, but rendered a work of this kind exceedingly necessary, lest in the general wreck of superstition, of false systems of government, and false theology, we lose sight of morality, of humanity, and of the theology that is true.\\nI believe in one God, and no more; and I hope for happiness beyond this life.\\nI believe in the equality of man; and I believe that religious duties consist in doing justice, loving mercy, and endeavoring to make our fellow-creatures happy.\\nI do not believe in the creed professed by the Jewish church, by the Roman church, by the Greek church, by the Turkish church, by the Protestant church, nor by any church that I know of. My own mind is my own church.\\nAll national institutions of churches, whether Jewish, Christian or Turkish, appear to me no other than human inventions, set up to terrify and enslave mankind, and monopolize power and profit.\\nI do not mean by this declaration to condemn those who believe otherwise; they have the same right to their belief as I have to mine.\\n—Thomas Paine, The Age of Reason, 1794–1795\\nPaine's statement in the last sentence, criticizing persecution, was most likely influenced by which of the following?\",\"src\":\"ori_mmlu-high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}