{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f50454e-9e33-507b-8f8a-b58d8dced93d","task_key":"default--test--5554","task_revision_id":"1","upstream_id":"5554","short_description":"Assume that the marginal propensity to consume increased from 0.75 in year 1 to…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['From year 1 to year 2, the spending multiplier decreased from 4 to 3.', 'In year 2, a $100 billion increase in government spending would have increased real output by a maximum of $75 billion.', 'From year 1 to year 2, the spending multiplier increased from 4 to 5.', 'The marginal propensity to save increased from year 1 to year 2.', 'A given change in government spending would have had a less powerful effect in year 1 than in year 2.', 'In year 2, a $100 billion increase in government spending would have decreased real output by a maximum of $80 billion.', 'The marginal propensity to consume decreased from 0.75 in year 1 to 0.70 in year 2.', 'The marginal propensity to save decreased from year 1 to year 2.', 'In year 1, a $100 billion increase in government spending would have increased real output by a maximum of $75 billion.']\",\"question\":\"Assume that the marginal propensity to consume increased from 0.75 in year 1 to 0.80 in year 2. Based on this information, which of the following is a correct conclusion?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}