{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcf86840-f754-58ba-9d5b-240deb82da3e","task_key":"default--test--6828","task_revision_id":"1","upstream_id":"6828","short_description":"Which of the following is a tool used by the Fed to increase the money supply?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decreasing government spending.', 'A lower discount rate.', 'A higher corporate tax rate.', 'Selling foreign exchange reserves.', 'Selling Treasury securities to commercial banks.', 'Decreasing the minimum wage.', 'Selling government bonds to the public.', 'Increasing interest rates.', 'A lower personal income tax rate.', 'A higher reserve ratio.']\",\"question\":\"Which of the following is a tool used by the Fed to increase the money supply?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}