{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c8b8067-e14e-58ae-9f24-8c39803c2dc7","task_key":"default--test--7106","task_revision_id":"1","upstream_id":"7106","short_description":"Which of the following lessens the impact of expansionary fiscal policy?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A decrease in government spending.', 'Lower interest rates that increase private investment.', 'Higher interest rates that decrease private investment.', 'Higher interest rates that cause an increase in net exports.', 'An increase in the marginal propensity to consume.', 'Lower taxes that increase consumer spending.', 'Lower interest rates that cause a decrease in net exports.', 'Lower inflation rates that increase net exports.']\",\"question\":\"Which of the following lessens the impact of expansionary fiscal policy?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}