{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab3114c5-d034-5cba-98ab-2cb398cdebf9","task_key":"default--test--7355","task_revision_id":"1","upstream_id":"7355","short_description":"If the Federal Reserve was concerned about the \"crowding-out\" effect they could…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['expansionary monetary policy by lowering the discount rate.', 'expansionary monetary policy by selling Treasury securities.', 'contractionary monetary policy by raising the discount rate.', 'contractionary monetary policy by lowering the discount rate.']\",\"question\":\"If the Federal Reserve was concerned about the \\\"crowding-out\\\" effect they could engage in\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}