{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64b2e340-cfd6-5d6b-820e-f6a3508ae71a","task_key":"all--test--64b2e340-cfd6-5d6b-820e-f6a3508ae71a","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Contemporary Deist spiritual beliefs were misguided in that they abandoned the Calvinist notions of \\\"original sin.\\\"', 'The United States had overestimated the good will and honor of Great Britain in terms of following the stipulations of the Treaty of Paris (1783).', 'The U.S. Army misread the willingness of American Indians in the Ohio Valley and Great Lakes regions to live side-by-side with white settlers.', 'The framers of the Articles of Confederation made a mistake in allowing for too great a degree of democracy in the new republic.']\",\"question\":\"This question refers to the following information.\\n\\\"Your sentiments, that our affairs are drawing rapidly to a crisis, accord with my own. What the event will be is also beyond the reach of my foresight. We have errors to correct. We have probably had too good an opinion of human nature in forming our confederation. Experience has taught us that men will not adopt and carry into execution measures the best calculated for their own good without the intervention of a coercive power. I do not conceive that we can exist long as a nation without having lodged somewhere a power which will pervade the whole Union in as energetic a manner as the authority of the state governments extends over the several states. . . .\\n\\\"What astonishing changes a few years are capable of producing. I am told that even respectable characters speak of a monarchical form of government without horror. . . . What a triumph for our enemies to verify their predictions! What a triumph for the advocates of despotism to find that we are incapable of governing ourselves, and that systems founded on the basis of equal liberty are merely ideal and fallacious. . . .\\\"\\n—George Washington, letter to John Jay, August 1, 1786\\nBased on the context of the letter, which of the following most closely describes the meaning of Washington's phrase, \\\"We have probably had too good an opinion of human nature\\\"?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}