{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b8cc5b8-b3d5-598b-9908-42aca44fcb8a","task_key":"default--test--7035","task_revision_id":"1","upstream_id":"7035","short_description":"A contractionary monetary policy will cause the nominal interest rate aggregate…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Increase', 'Increase \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Increase', 'Increase    Increase    Decrease    Decrease', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Increase \\\\xa0\\\\xa0\\\\xa0 Increase', 'Decrease    Increase    Decrease    Increase', 'Increase    Increase    Decrease    Increase', 'Decrease    Decrease    Decrease    Decrease', 'Increase \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Decrease    Decrease    Increase    Decrease', 'Increase    Increase    Increase    Decrease']\",\"question\":\"A contractionary monetary policy will cause the nominal interest rate aggregate demand output and the price level to change in which of the following ways? NOMINAL INTEREST RATE     AGGREGATE DEMAND     OUTPUT     PRICE LEVEL\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}