{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b1cebb4-3906-5bec-9845-1f669c654b57","task_key":"default--test--7302","task_revision_id":"1","upstream_id":"7302","short_description":"Production possibilities frontiers are concave to the origin because","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['of the principle of comparative advantage.', 'of the existence of fixed resources.', 'of the scarcity of resources.', 'of the law of decreasing costs.', 'of inefficiencies in the economy.', 'of the law of demand and supply.', 'of the law of increasing costs.', 'of constant opportunity costs.', 'of economies of scale.', 'of opportunity cost.']\",\"question\":\"Production possibilities frontiers are concave to the origin because\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}