{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6da87f62-482d-5abb-a94a-fc0978d442c5","task_key":"default--test--7347","task_revision_id":"1","upstream_id":"7347","short_description":"Suppose that an expansionary fiscal policy leads to a large increase in real…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['inflation had already impacted the economy before the fiscal stimulus.', 'aggregate demand is steeply sloped.', 'the fiscal policy was ineffective.', 'aggregate supply decreased.', 'aggregate supply increased.', 'the economy initially had some unemployed resources.', 'the economy was operating at full employment before the fiscal stimulus.', 'the fiscal policy led to deflation.', 'aggregate demand is perfectly elastic.']\",\"question\":\"Suppose that an expansionary fiscal policy leads to a large increase in real output and a small increase in the price level. From this it can be inferred that\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}