{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"caa2751c-0a07-5504-8fa1-6720bee6e137","task_key":"default--test--6838","task_revision_id":"1","upstream_id":"6838","short_description":"When a firm is earning a normal profit from the production of a good, it is true…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['total revenues from production are equal to implicit costs.', 'total revenues from production are equal to explicit costs.', 'explicit costs are higher than implicit costs.', 'total revenues from production are equal to the sum of explicit and implicit costs.', 'total revenues from production are lower than the sum of explicit and implicit costs.', 'explicit costs are equal to implicit costs.', 'implicit costs are higher than explicit costs.', 'total revenues from production are higher than the sum of explicit and implicit costs.', 'total revenues from production are equal to double the sum of explicit and implicit costs.', 'implicit costs are equal to the difference of total revenues from production and explicit costs.']\",\"question\":\"When a firm is earning a normal profit from the production of a good, it is true that\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}