{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4111844b-ad94-5a03-8291-c640a520755c","task_key":"default--test--7166","task_revision_id":"1","upstream_id":"7166","short_description":"Suppose the Japanese economy is suffering a prolonged recession. Lower Japanese…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decrease     Increase     Increase', 'No Change     Decrease     Increase', 'Increase     Increase     Decrease', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Decrease     No Change     Decrease', 'Increase     Decrease     Increase', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Increase \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Increase', 'Increase     Increase     Increase']\",\"question\":\"Suppose the Japanese economy is suffering a prolonged recession. Lower Japanese household incomes will affect U.S. exports to Japan demand for the dollar and the value of the dollar relative to the yen in which of the following ways? EXPORTS TO JAPAN     DEMAND FOR $     VALUE OF $\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}