{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96c4a231-2f31-5e0e-af03-1cefd44cf501","task_key":"all--test--96c4a231-2f31-5e0e-af03-1cefd44cf501","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The First Great Awakening', 'The natural rights theory of John Locke', 'British ideas about social hierarchy', 'Eighteenth century scientific racism']\",\"question\":\"This question refers to the following information.\\nOn Being Brought from Africa to America\\n'Twas mercy brought me from my Pagan land,\\nTaught my benighted soul to understand\\nThat there's a God, that there's a Saviour too;\\nOnce I redemption neither sought nor knew.\\nSome view our sable race with scornful eye,\\n\\\"Their colour is a diabolic die.\\\"\\nRemember, Christians, Negroes, black as Cain,\\nMay be refin'd, and join th' angelic train.\\n—Phillis Wheatley, Poems on Various Subjects, Religious and Moral, 1773\\nThe ideas expressed in Phillis Wheatley's poem most directly reveal the influence of which of the following?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}