{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75f20757-544c-51eb-9087-b44e13d12268","task_key":"default--test--7354","task_revision_id":"1","upstream_id":"7354","short_description":"Of the following choices the one most likely to be preferred by supply-side…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['lower taxes on household income.', 'a tax credit on capital investment.', 'increased government spending.', 'higher tariffs on imported goods.', 'A tax on wealth', 'Government regulation of industry', 'Increased minimum wage', 'Higher corporate tax rates', 'Reduction in defense spending', 'Increased social welfare programs']\",\"question\":\"Of the following choices the one most likely to be preferred by supply-side economists would be\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}