{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03c1f654-4a9e-529d-8b5d-d067bff30571","task_key":"default--test--7478","task_revision_id":"1","upstream_id":"7478","short_description":"What were some causes of the stagflation in the early 1970' s?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decrease in unemployment', 'Rapid acceleration of price level, oil price increases, perversely working tax systems, and macroeconomic policy mistakes', 'Lowering of oil prices', 'Expansion of international trade without barriers', 'Increase in the gold standard value', 'Excessive production of goods', 'Advancements in technology reducing costs', 'Stabilization of currency exchange rates', 'Economic expansion', 'Reduction in government spending']\",\"question\":\"What were some causes of the stagflation in the early 1970' s?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}