{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1681b46f-3e60-5f63-814f-bf7f6c3c7c35","task_key":"default--test--7640","task_revision_id":"1","upstream_id":"7640","short_description":"Suppose taxes are cut in an economy that is in equilibrium at full employment.…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['have no effect on the price level but raise real output.', 'lower real output without affecting the price level.', 'lower the price level.', 'have no effect on real output or the price level.', 'raise real output and lower the price level.', 'lower real output and lower the price level.', 'raise the price level.', 'raise real output without affecting the price level.', 'raise real output and raise the price level.', 'lower real output and raise the price level.']\",\"question\":\"Suppose taxes are cut in an economy that is in equilibrium at full employment. In the long run the tax cut will\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}