{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30c3a8d9-de9e-5357-a438-4348e786b905","task_key":"all--test--30c3a8d9-de9e-5357-a438-4348e786b905","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['political leaders in the 1960s and 1970s did not do enough to challenge the racial and economic status quo.', 'the United States had relied too heavily on military solutions and needed to pursue a more isolationist foreign policy.', 'religious leaders had become too assertive in governmental policy and had blurred the separation of church and state.', 'liberal laws and court decisions in the 1960s and a general moral decline were undermining the United States.']\",\"question\":\"This question refers to the following information.\\n\\\"The economic ills we suffer have come upon us over several decades. They will not go away in days, weeks, or months, but they will go away. They will go away because we as Americans have the capacity now, as we've had in the past, to do whatever needs to be done to preserve this last and greatest bastion of freedom. In this present crisis, government is not the solution to our problem; government is the problem. . . .\\n\\\"It is my intention to curb the size and influence of the Federal establishment and to demand recognition of the distinction between the powers granted to the Federal Government and those reserved to the States or to the people. All of us need to be reminded that the Federal Government did not create the States; the States created the Federal Government.\\\"\\n—Ronald Reagan, First Inaugural Address, January 21, 1981\\nThe conservative political revival that led to President Ronald Reagan's election in 1980 held the view that\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}