{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"846a2482-c2c1-5b30-9891-6e612c465547","task_key":"default--test--7493","task_revision_id":"1","upstream_id":"7493","short_description":"There is relatively more crowding out as the result of expansionary fiscal…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the central bank pursues a contractionary monetary policy', 'expansionary monetary policy accompanies the fiscal policy', 'the government cuts taxes rather than increasing spending', 'the investment demand curve is perfectly inelastic', 'government spending improves profit expectations among businesses', 'the investment demand curve is inelastic', 'the investment demand curve is elastic', 'the economy is in a boom period', 'the economy is in a recession', 'government spending is primarily focused on infrastructure']\",\"question\":\"There is relatively more crowding out as the result of expansionary fiscal policy when\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}