{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3761b327-7fbc-5081-803a-73993f4a6d78","task_key":"default--test--7639","task_revision_id":"1","upstream_id":"7639","short_description":"As the Japanese economy expands in what ways do U.S. net exports the values of…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decrease \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Increase \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Increase', 'Decrease \\\\xa0\\\\xa0\\\\xa0 No Change \\\\xa0\\\\xa0\\\\xa0 Increase', 'Increase \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Increase', 'No Change \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Increase \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Increase', 'Increase \\\\xa0\\\\xa0\\\\xa0 No Change \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Increase', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease']\",\"question\":\"As the Japanese economy expands in what ways do U.S. net exports the values of the dollar and the yen change? U.S. NET EXPORTS     VALUE OF DOLLAR     VALUE OF YEN\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}