{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24b6e997-1c9e-5ab3-9a01-cbf690ec2788","task_key":"default--test--7428","task_revision_id":"1","upstream_id":"7428","short_description":"If current real GDP is $5000 and full employment real GDP is at $4000 which of…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A decrease in taxes and buying bonds in an open market operation', 'An increase in government spending and an increase in taxes', 'An increase in taxes and selling bonds in an open market operation', 'An increase in taxes and a lower discount rate', 'A decrease in taxes and selling bonds in an open market operation', 'A decrease in taxes and a lower discount rate', 'An increase in government spending and a decrease in taxes', 'A decrease in government spending and an increase in the discount rate', 'A decrease in government spending and a decrease in the discount rate', 'An increase in government spending and an increase in the discount rate']\",\"question\":\"If current real GDP is $5000 and full employment real GDP is at $4000 which of the following combinations of policies is the most likely to have brought the economy to this point?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}