{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da25cae1-dbd6-51c3-afe4-56fa2b962225","task_key":"default--test--6970","task_revision_id":"1","upstream_id":"6970","short_description":"In a private closed economy which of the following statements is true?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Investment is always less than savings.', 'Saving is equal to zero when consumption equals disposable income.', 'Investment is always greater than savings.', 'Saving is equal to zero when consumption exceeds disposable income.', 'The economy is in equilibrium when consumption equals saving.', 'The economy is in equilibrium when consumption exceeds saving.', 'Household saving can occasionally be negative.', 'Household saving is always equal to investment.', 'The economy is in equilibrium when investment equals saving.', 'Household saving can never be negative.']\",\"question\":\"In a private closed economy which of the following statements is true?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}