{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ea58333e-3cea-5096-99de-c820dfb1fb75","task_key":"default--test--6079","task_revision_id":"1","upstream_id":"6079","short_description":"The World Systems Theory divides the world as a unit into three subcategories:…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Small percentage of very high class individuals.', 'Highly industrialized and technologically advanced nations.', 'Countries with a high rate of population growth.', 'Strong central government.', 'High rates of urbanization.', 'Economies dependent on foreign aid.', 'Low income, agrarian societies.', 'Countries with no natural resources.', 'A diversified and developed economy.', 'An economy centralized around one natural resource.']\",\"question\":\"The World Systems Theory divides the world as a unit into three subcategories: core countries, periphery countries, and semi-periphery countries. Which of the following answers correlates best with semi-periphery countries?\",\"src\":\"ori_mmlu-college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}