{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7f86c74-ef25-5be4-9fc0-3fe313bea321","task_key":"default--test--7214","task_revision_id":"1","upstream_id":"7214","short_description":"What is the effect of increasing costs (decreasing returns to scale) on…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Increasing costs have no effect on specialization', 'Increasing costs force companies to merge, reducing the need for specialization', 'Increasing costs enhance the degree of specialization', 'Increasing costs result in higher levels of specialization to spread out the costs', 'Increasing costs limit the degree of specialization', 'Increasing costs lead to a decrease in market competition, which promotes specialization', 'Increasing costs lead to technological innovation, which increases specialization', 'Increasing costs promotes diversification instead of specialization']\",\"question\":\"What is the effect of increasing costs (decreasing returns to scale) on specialization?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}