{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e55fb77a-ee46-59e2-9261-ef624dc5facd","task_key":"all--test--e55fb77a-ee46-59e2-9261-ef624dc5facd","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Southern Baptist justification of slavery on the grounds of white racial superiority', 'the Puritan abolition of slavery in New England states', 'Jewish acceptance of slavery in the Torah', 'Christian abolitionist rejection of slavery on moral grounds']\",\"question\":\"This question refers to the following information.\\n\\\"On the 4th of March next this party [the Republican party] will take possession of the government. It has announced that the South shall be excluded from the common territory, that the judicial tribunal shall be made sectional, and that a war must be waged against slavery until it shall cease throughout the United States. The guarantees of the Constitution will then no longer exist; the equal rights of the states will be lost. The slaveholding states will no longer have the power of self-government or self-protection, and the federal government will have become their enemy.\\n\\\"We, therefore, the people of South Carolina, by our delegates in convention assembled, have solemnly declared that the Union heretofore existing between this state and the other states of North America is dissolved; and that the state of South Carolina has resumed her position among the nations of the world, as [a] separate and independent state, with full power to levy war, conclude peace, contract alliances, establish commerce, and to do all other acts and things which independent states may of right do.\\\"\\nSouth Carolina defines the causes of secession, 1860\\nIn the excerpt above, the reference to \\\"the sanctions of a more erroneous religious belief\\\" most probably refers to\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}