{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77f85e96-321e-5193-bb24-352bfdf002c2","task_key":"default--test--6891","task_revision_id":"1","upstream_id":"6891","short_description":"Which of the following statements is true?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The demand for loanable funds consists of savers.', 'Real GDP is the sum of the money supply and the velocity of money.', \\\"The supply of loanable funds is solely determined by the central bank's policies.\\\", 'Dollars earned today have more purchasing power than dollars earned a year from today.', 'The velocity of money is the speed at which the stock market indices change.', 'The demand for loanable funds is entirely driven by government borrowing.', 'Purchasing power parity ensures that dollars have the same value across all time periods.', 'The supply of loanable funds consists of investors.', 'Inflation rates are inversely proportional to the velocity of money at all times.']\",\"question\":\"Which of the following statements is true?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}