{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d0c91b4-9261-5d98-a2f8-25f0e377bc1a","task_key":"default--test--4725","task_revision_id":"1","upstream_id":"4725","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Renaissance humanism', 'Empiricism', 'Aristotelianism', 'Byzantinism', 'Stoicism', 'Scholasticism', 'Existentialism', 'Neoplatonism', 'Neoclassicism', 'Rationalism']\",\"question\":\"This question refers to the following information.\\n\\\"The substitution of Plato for the scholastic Aristotle was hastened by contact with Byzantine scholarship. Already at the Council of Ferrera (1438), which nominally reunited the Eastern and Western churches, there was a debate in which the Byzantines maintained the superiority of Plato to Aristotle. Cosimo and Lorenzo de Medici were both addicted to Plato; Cosimo founded and Lorenzo continued the Florentine Academy, which was largely devoted to the study of Plato&…The humanists of the time, however, were too busy acquiring knowledge of antiquity to be able to produce anything of value.\\\"\\nBertrand Russell, British philosopher, History of Western Philosophy, 1946\\nAccording to the information in the text, the schools founded by the Medici family most favored a philosophy known as\",\"src\":\"ori_mmlu-high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}