{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88e38221-b4cb-5a85-b3d5-768021641ca8","task_key":"all--test--88e38221-b4cb-5a85-b3d5-768021641ca8","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['President Lincoln was a consistent supporter of the abolitionist cause.', 'President Lincoln was reluctant to be ideologically associated with advocates like Frederick Douglass.', 'In his ambition to become president, Abraham Lincoln declared his desire to use his constitutional powers to end slavery.', 'President Lincoln had continually changing views on slavery and abolition that did not always fit into the prevailing political categories.']\",\"question\":\"This question refers to the following information.\\n\\\"Lincoln was strongly anti-slavery, but he was not an abolitionist or a Radical Republican and never claimed to be one. He made a sharp distinction between his frequently reiterated personal wish that 'all men everywhere could be free' and his official duties as a legislator, congressman, and president in a legal and constitutional system that recognized the South's right to property in slaves. Even after issuing the Emancipation Proclamation he continued to declare his preference for gradual abolition. While his racial views changed during the Civil War, he never became a principled egalitarian in the manner of abolitionists such as Frederick Douglass or Wendell Phillips or Radical Republicans like Charles Sumner.\\\"\\n—Eric Foner, The Fiery Trial, 2010\\nWhich of the following statements best describes Eric Foner's argument about President Abraham Lincoln's views on slavery?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}