{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67035629-4c15-5c30-9878-84a83dad3e30","task_key":"default--test--7292","task_revision_id":"1","upstream_id":"7292","short_description":"In the short run contractionary monetary policy causes aggregate demand to…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Stays the same \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases', 'Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases', 'Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases', 'Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases', 'Stays the same \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases', 'Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases', 'Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases', 'Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Increases', 'Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases', 'Increases \\\\xa0\\\\xa0\\\\xa0\\\\xa0  \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0\\\\xa0 Decreases']\",\"question\":\"In the short run contractionary monetary policy causes aggregate demand to output to  and the price level to Aggregate demand      Output      Price Level\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}