{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2893002c-be47-57f8-a32a-187e77b3ecf8","task_key":"default--test--7574","task_revision_id":"1","upstream_id":"7574","short_description":"Which of the following statements is positive?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The law of increasing costs leads to a downward sloping production possibilities frontier.', 'Nations should concentrate their resources on producing wholesome consumer goods as opposed to the weapons of war.', 'The production possibilities frontier is linear due to constant costs.', 'The production possibilities frontier is convex to the origin because of the law of decreasing costs.', 'An economy that produces more butter than guns is better off than an economy that produces more guns than butter.', 'The law of diminishing returns applies to the production of both consumer goods and weapons.', 'Nations ought to devote at least some of their resources to national defense.', 'Nations should prioritize military production to ensure their national security.', 'An economy that produces more guns than butter is worse off than an economy that produces more butter than guns.', 'The production possibilities frontier is concave to the origin because of the law of increasing costs.']\",\"question\":\"Which of the following statements is positive?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}