{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc4202a8-893d-5e56-8a9b-4acf34db892c","task_key":"default--test--7646","task_revision_id":"1","upstream_id":"7646","short_description":"A monopoly is less efficient than a perfect competitor because","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['a monopoly can make profit in the short run but not in the long run', 'a monopoly can control the supply and demand of the market', 'a perfect competitor can make profit in the short run but the monopoly cannot', 'a perfect competitor breaks even in the short run and the monopoly does not', 'a monopoly sells for a lower price and produces more output', 'a monopoly produces more output and sells for a higher price', 'a monopoly has no competition, therefore it is less efficient', 'a perfect competitor produces more output and sells for a lower price', 'a perfect competitor breaks even in the long run and the monopoly does not', 'a monopoly produces less output and sells for a higher price']\",\"question\":\"A monopoly is less efficient than a perfect competitor because\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}