{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba4cdcae-d7f5-5852-ada0-e004521784ab","task_key":"all--test--ba4cdcae-d7f5-5852-ada0-e004521784ab","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The Declaration of Independence', 'Jonathan Edwards\\\\'s sermon \\\"Sinners in the Hands of an Angry God\\\"', 'The Seneca Falls Declaration of Rights and Sentiments', 'Martin Luther King, Jr.\\\\'s \\\"I Have a Dream” speech']\",\"question\":\"This question refers to the following information.\\nOn Being Brought from Africa to America\\n'Twas mercy brought me from my Pagan land,\\nTaught my benighted soul to understand\\nThat there's a God, that there's a Saviour too;\\nOnce I redemption neither sought nor knew.\\nSome view our sable race with scornful eye,\\n\\\"Their colour is a diabolic die.\\\"\\nRemember, Christians, Negroes, black as Cain,\\nMay be refin'd, and join th' angelic train.\\n—Phillis Wheatley, Poems on Various Subjects, Religious and Moral, 1773\\nThe point of Wheatley's poem can best be compared to which of the following?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}