{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23f4b160-d8ed-51d5-8323-bb85bb8ac13d","task_key":"default--test--7421","task_revision_id":"1","upstream_id":"7421","short_description":"U.S. dollars and the European Union's (EU's) euro are exchanged in global…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['If inflation is high in the EU and the price level in the United States is stable the value of the dollar appreciates.', 'If the Fed decreases the money supply the value of the dollar depreciates.', 'If EU consumers are less inclined to purchase American goods the dollar appreciates.', 'If U.S. income levels are rising relative to incomes in the EU the euro depreciates.']\",\"question\":\"U.S. dollars and the European Union's (EU's) euro are exchanged in global currency markets. Which of the following is true?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}