{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6a9d670-5aab-5561-9637-b0a45f211a36","task_key":"default--test--7096","task_revision_id":"1","upstream_id":"7096","short_description":"Which of the following fiscal policies is likely to be most effective when the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The government decreases both taxes and spending.', 'The government keeps taxes and spending unchanged.', 'The government increases spending and keeps taxes unchanged.', 'The government increases spending matched with an increase in taxes.', 'The government decreases taxes and increases spending.', 'The government increases taxes and decreases spending.', 'The government increases both taxes and spending.', 'The government decreases taxes and keeps spending unchanged.', 'The government decreases spending and keeps taxes unchanged.', 'The government keeps spending unchanged and decreases taxes.']\",\"question\":\"Which of the following fiscal policies is likely to be most effective when the economy is experiencing an inflationary gap?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}