{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74bd795d-84c5-546f-8616-eda67d9094e6","task_key":"all--test--74bd795d-84c5-546f-8616-eda67d9094e6","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Dred Scott v. Sanford (1857).', 'Brown v.Board of Education of Topeka, Kansas (1954).', 'Worcester v. Georgia (1831).', 'Plessy v. Ferguson (1896).']\",\"question\":\"This question refers to the following information.\\n\\\"As a means of effecting this end I suggest for your consideration the propriety of setting apart an ample district west of the Mississippi, and without the limit of any State or Territory now formed, to be guaranteed to the Indian tribes as long as they shall occupy it. . . . There they may be secured in the enjoyment of governments of their own choice, subject to no other control from the United States than such as may be necessary to preserve peace on the frontier and between the several tribes. There the benevolent may endeavor to teach them the arts of civilization. . . .\\n\\\"This emigration would be voluntary, for it would be as cruel and unjust to compel the aborigines to abandon the graves of their fathers and seek a home in a distant land. But they should be distinctly informed that if they remain within the limits of the States they must be subject to their laws. . . .\\\"\\n—President Andrew Jackson, 1829\\nWhich of the following Supreme Court decisions was the primary contributor to the system of segregation that Martin Luther King and others were trying to end?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}