{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c84274ce-449b-58ec-b3d1-2d73213779a1","task_key":"default--test--7285","task_revision_id":"1","upstream_id":"7285","short_description":"If the economy is operating at full employment which of the following policies…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['An increase in taxes with no change in government spending', 'No change in government spending or taxes', 'An increase in government spending matched by an equal increase in taxes', 'A decrease in government spending and a matching increase in taxes', 'An increase in government spending with no change in taxes', 'A decrease in taxes with an increase in government spending', 'A decrease in government spending with no change in taxes', 'An increase in taxes and a decrease in government spending', 'A decrease in taxes with no change in government spending', 'A decrease in taxes and an increase in government spending']\",\"question\":\"If the economy is operating at full employment which of the following policies will create the most inflation in the short run?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}