{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f43d2e38-58e6-5753-b512-e4c88e2dd83b","task_key":"all--test--f43d2e38-58e6-5753-b512-e4c88e2dd83b","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['American economic prosperity.', \\\"President Ronald Reagan's Strategic Defense Initiative.\\\", 'the Iranian Hostage Crisis.', 'the end of the Cold War.']\",\"question\":\"This question refers to the following information.\\nClearly, no longer can a dictator count on East–West confrontation to stymie concerted United Nations action against aggression. A new partnership of nations has begun. And we stand today at a unique and extraordinary moment. The crisis in the Persian Gulf, as grave as it is, also offers a rare opportunity to move toward an historic period of cooperation. Out of these troubled times, our fifth objective—a new world order—can emerge; a new era, freer from the threat of terror, stronger in the pursuit of justice, and more secure in the quest for peace. An era in which the nations of the world, east and west, north and south, can prosper and live in harmony.\\n—President George Herbert Walker Bush, Speech to a Joint Session of Congress, September 11, 1990\\nPresident George Herbert Walker Bush's vision of a \\\"new world order\\\" was most directly made possible by\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}