{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a3e7885-1bbe-544e-ab84-c2c19339707a","task_key":"default--test--6957","task_revision_id":"1","upstream_id":"6957","short_description":"The effect of the spending multiplier is lessened if","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the price level rises with a decrease in aggregate demand.', 'the price level is constant with a decrease in aggregate demand.', 'the price level rises with an increase in aggregate demand.', 'the price level is constant with a decrease in long-run aggregate supply.', 'the price level is constant with an increase in aggregate demand.', 'the price level falls with an increase in aggregate supply.', 'the price level is constant with an increase in short-run aggregate supply.', 'the price level is constant with an increase in long-run aggregate supply.', 'the price level decreases with a decrease in aggregate demand.', 'the price level falls with a decrease in aggregate supply.']\",\"question\":\"The effect of the spending multiplier is lessened if\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}