{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fcccad77-20ca-5d48-9837-15c0481ce4a1","task_key":"all--test--fcccad77-20ca-5d48-9837-15c0481ce4a1","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Lincoln received more votes in the heavily populated states of the North, while McClellan won more sparsely populated states.', 'Lincoln won the Election of 1864 due to Union successes during the Civil War.', 'The Emancipation Proclamation mobilized black voters in the South to vote for Lincoln in the Election of 1864.', 'Republicans managed to gain control of Congress through the efforts of Lincoln to campaign on their behalf.']\",\"question\":\"This question refers to the following information.\\n\\\"With 78 percent of the Union electorate casting ballots, Lincoln was reelected in an Electoral College landslide, 212 to McClellan's 21. The 55% popular vote for the president was the third largest in the nineteenth century, surpassed only by Jackson's first victory in 1828 and Grant's reelection in 1872. McClellan won only New Jersey, Delaware, and Kentucky. Republicans drew support from native-born farmers, skilled and professional workers, those of New England descent, younger voters, and military personnel. Democrats were strongest in the cities and among Irish- and German-Americans (the most populous immigrant groups). It has been estimated that Lincoln received 78% of the vote of Union soldiers and sailors. The figure was not necessary for his reelection, but was perhaps the margin of victory in a few close states and, more importantly, of great symbolic value. Republicans also gained seats in Congress to retain unassailable control, 149 to 42 in the House and 42 to 10 in the Senate; took back several state legislatures; and lost only the governorship of New Jersey (McClellan's home state).\\\"\\nElection of 1864, Harpweek.com\\nWhich of the following conclusions is best supported by the excerpt above?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}