{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b37f09a1-8675-567c-b5cf-e3dc3049cab7","task_key":"all--test--b37f09a1-8675-567c-b5cf-e3dc3049cab7","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Debates over the powers of the president.', 'Debates over participation in elections.', 'Debates over federal power over the economy.', 'Debates over federal power over international affairs.']\",\"question\":\"This question refers to the following information.\\n\\\"The economic ills we suffer have come upon us over several decades. They will not go away in days, weeks, or months, but they will go away. They will go away because we as Americans have the capacity now, as we've had in the past, to do whatever needs to be done to preserve this last and greatest bastion of freedom. In this present crisis, government is not the solution to our problem; government is the problem. . . .\\n\\\"It is my intention to curb the size and influence of the Federal establishment and to demand recognition of the distinction between the powers granted to the Federal Government and those reserved to the States or to the people. All of us need to be reminded that the Federal Government did not create the States; the States created the Federal Government.\\\"\\n—Ronald Reagan, First Inaugural Address, January 21, 1981\\nWhich of the following ongoing debates in U.S. history is expressed in the quotation?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}