{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"908394e0-88ed-5040-8f73-f88cb1a00a84","task_key":"default--test--6959","task_revision_id":"1","upstream_id":"6959","short_description":"The appropriate fiscal policy to remedy a recession is to","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['increase government spending and taxes.', 'reduce government spending and increase taxes.', 'increase taxes without changing government spending.', 'maintain current levels of government spending and taxes.', 'reduce government spending and taxes.', 'reduce taxes without changing government spending.', 'increase government spending and reduce taxes.', 'decrease government spending and increase taxes.', 'increase government spending without changing tax levels.', 'reduce government spending without changing tax levels.']\",\"question\":\"The appropriate fiscal policy to remedy a recession is to\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}