{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c16c5fe9-ec41-5159-8873-a61824470766","task_key":"default--test--7284","task_revision_id":"1","upstream_id":"7284","short_description":"The economy is currently operating at full employment. Assuming flexible wages…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Falls \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 Falls', 'No change \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 No change', 'Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 Falls', 'No change \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls', 'Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls', 'Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 No change', 'Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 No change', 'No change \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 No change', 'Falls \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls', 'No change \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 No change \\\\xa0\\\\xa0\\\\xa0 Falls']\",\"question\":\"The economy is currently operating at full employment. Assuming flexible wages and prices how would a decline in aggregate demand affect GDP and the price level in the short run and GDP and the price level in the long run? SHORT-RUN GDP     SHORT-RUN PRICE LEVEL     LONG-RUN GDP     LONG-RUN PRICE LEVEL\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}