{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42152bf1-6727-5cd5-896d-a690b44340bf","task_key":"default--test--7274","task_revision_id":"1","upstream_id":"7274","short_description":"What major microeconomic problems have plagued the Soviet planned economy?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"[\\\"Inefficient satisfaction of consumers' desires, emphasis on physical output obscuring quality and cost, resistance to superior technologies due to higher production quotas\\\", 'Overemphasis on consumer goods production at the expense of capital goods', 'Hyperinflation due to overprinting of currency', 'Lack of foreign trade', 'Complete autonomy of enterprises leading to market competition', 'Overemphasis on agricultural production', 'Reliance on imported goods leading to trade deficits', 'Excessive focus on service industries rather than manufacturing']\",\"question\":\"What major microeconomic problems have plagued the Soviet planned economy?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}