{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be38d061-dc73-5e0f-8b09-0d71e0a16026","task_key":"default--test--6887","task_revision_id":"1","upstream_id":"6887","short_description":"The money supply will increase as a result of which of the following?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['An increase in the fraction of deposits that must be held by banks', 'The purchase of bonds by private individuals', 'An increase in the amount of physical cash in circulation', 'A decrease in the amount of money people are saving', 'The Federal Reserve raising interest rates', 'The selling of bonds by the Federal Reserve', 'An increase in the discount rate', 'The government increasing taxes', 'A decrease in the required reserve ratio', 'A decrease in the rate of inflation']\",\"question\":\"The money supply will increase as a result of which of the following?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}