{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94f26110-8037-58df-9fca-9f6c0ed4f1c8","task_key":"default--test--7484","task_revision_id":"1","upstream_id":"7484","short_description":"If the Japanese economy suffers a deep prolonged recession in what ways would…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Increases    Increases    Increases', 'Decreases \\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0 Increases', 'Increases    Increases    Decreases', 'No change    Increases    Increases', 'Decreases    Increases    Decreases', 'Increases    Decreases    Decreases', 'No change    Decreases    Decreases', 'Decreases \\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0 Decreases', 'Decreases \\\\xa0\\\\xa0\\\\xa0 Increases \\\\xa0\\\\xa0\\\\xa0 Increases', 'Increases \\\\xa0\\\\xa0\\\\xa0 Decreases \\\\xa0\\\\xa0\\\\xa0 Increases']\",\"question\":\"If the Japanese economy suffers a deep prolonged recession in what ways would U.S. net exports and the values of the dollar and yen change? U.S. NET EXPORTS     VALUE OF DOLLAR     VALUE OF YEN\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}