{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c38528d-beb1-57c0-8b9a-740fade14a3d","task_key":"default--test--7441","task_revision_id":"1","upstream_id":"7441","short_description":"If the government regulates a monopoly to produce at the allocative efficient…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The monopoly would make an economic profit.', 'The monopoly would break even.', 'The monopoly would increase its prices.', 'The deadweight loss in this market would increase.', 'The deadweight loss in this market would decrease.', 'The consumer surplus in this market would decrease.']\",\"question\":\"If the government regulates a monopoly to produce at the allocative efficient quantity, which of the following would be true?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}