{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce3c88e3-05b0-52ef-8442-09746dea8913","task_key":"default--test--7290","task_revision_id":"1","upstream_id":"7290","short_description":"Which of the following choices is most likely to create the greatest decrease in…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The government decreases both spending and taxes.', 'The government holds spending and taxes constant.', 'The government decreases spending matched with a decrease in taxes.', 'The government decreases taxes with no change in spending.', 'The government holds spending constant while increasing taxes.', 'The government increases both spending and taxes.', 'The government increases spending with a corresponding increase in taxes.', 'The government increases taxes with no increase in spending.', 'The government increases spending with no increase in taxes.', 'The government decreases spending with no change in taxes.']\",\"question\":\"Which of the following choices is most likely to create the greatest decrease in real GDP?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}