{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83a635f5-a19a-5395-a5ee-4d2b8d3cdfb9","task_key":"default--test--7167","task_revision_id":"1","upstream_id":"7167","short_description":"Suppose the economy is in long-run equilibrium when a temporary expansionary…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Remains the same  \\\\xa0\\\\xa0\\\\xa0Falls \\\\xa0\\\\xa0\\\\xa0 Rises', 'Remains the same \\\\xa0\\\\xa0\\\\xa0 Rises \\\\xa0\\\\xa0\\\\xa0 Falls', 'Shifts up \\\\xa0\\\\xa0\\\\xa0 Rises \\\\xa0\\\\xa0\\\\xa0 Falls', 'Shifts down \\\\xa0\\\\xa0\\\\xa0 Rises \\\\xa0\\\\xa0\\\\xa0 Rises', 'Shifts down \\\\xa0\\\\xa0\\\\xa0 Rises \\\\xa0\\\\xa0\\\\xa0 Falls', 'Shifts up \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Rises', 'Shifts up \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls', 'Shifts down \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Falls', 'Shifts up \\\\xa0\\\\xa0\\\\xa0 Rises \\\\xa0\\\\xa0\\\\xa0 Rises', 'Shifts down \\\\xa0\\\\xa0\\\\xa0 Falls \\\\xa0\\\\xa0\\\\xa0 Rises']\",\"question\":\"Suppose the economy is in long-run equilibrium when a temporary expansionary supply shock is felt in the economy. This changes the short-run Phillips curve the short-run unemployment rate and the long-run unemployment rate in which of the following ways? SHORT-RUN PHILLIPS CURVE     SHORT-RUN UNEMPLOYMENT     LONG-RUN UNEMPLOYMENT\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}