{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7388bdb3-8ae1-5499-8324-4af37fbb590d","task_key":"default--test--7491","task_revision_id":"1","upstream_id":"7491","short_description":"It is a monetarist point of view that","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the money supply should be increased steadily', 'the velocity of money is unstable', 'crowding out is not a problem', 'inflation is primarily caused by the decrease in the money supply', 'money supply should be decreased to control inflation', 'monetary policy has no impact on the economy', 'fiscal policy is more effective than monetary policy', 'crowding out is the main problem in the economy', 'monetary policy should be used to fine-tune the economy']\",\"question\":\"It is a monetarist point of view that\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}