{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c25e5453-3720-56fe-bf24-568b9d43e6ec","task_key":"default--test--4719","task_revision_id":"1","upstream_id":"4719","short_description":"This question refers to the following information.","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Decreasing federal income tax rates in the 1980', 'The economic stimulus package during the 2008 financial crisis', 'The protective tariffs established during the Industrial Revolution', 'The implementation of the Gold Standard in the 19th century', 'The deregulation policies of the 1980s', 'The establishment of the Federal Reserve in 1913', 'Free trade policies of the 1970s', \\\"The New Deal's economic policies during the Great Depression\\\", 'Antitrust reforms of the Progressive Era', 'Increasing federal income tax rates after World War I']\",\"question\":\"This question refers to the following information.\\n\\\"Whereas it is expedient that new provisions and regulations should be established for improving the revenue of this kingdom, and for extending and securing the navigation and commerce between Great Britain and your Majesty's dominions in America, which, by the peace, have been so happily enlarged: and whereas it is just and necessary, that a revenue be raised, in your Majesty's said dominions in America, for defraying the expenses of defending, protecting, and securing the same; we, your Majesty's most dutiful and loyal subjects, the commons of Great Britain, in parliament assembled, being desirous to make some provision, in this present session of parliament, towards raising the said revenue in America, have resolved to give and grant unto your Majesty the several rates and duties herein after-mentioned….\\\"\\nThe Sugar Act of 1764\\nThe goals presented in the excerpt have the most in common with which of the following?\",\"src\":\"ori_mmlu-high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}