{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23cf0b23-1b9e-56f3-b382-2ecf67045983","task_key":"all--test--23cf0b23-1b9e-56f3-b382-2ecf67045983","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['As an immigrant, Carnegie had no right to own controlling interests in major American industries.', 'Carnegie did nothing to prevent the use of violence against his workers when they asked for better wages and working conditions.', 'Carnegie was seen as the epitome of American success, but he was not even the richest man in America.', \\\"Too much of Carnegie's philanthropic efforts were concentrated on his homeland of Scotland, denying Americans the benefits of his charity.\\\"]\",\"question\":\"This question refers to the following information.\\n\\\"One-half of the people of this nation to-day are utterly powerless to blot from the statute books an unjust law, or to write there a new and a just one. The women, dissatisfied as they are with this form of government, that enforces taxation without representation,—that compels them to obey laws to which they have never given their consent,—that imprisons and hangs them without a trial by a jury of their peers, that robs them, in marriage, of the custody of their own persons, wages and children,—are this half of the people left wholly at the mercy of the other half, in direct violation of the spirit and letter of the declarations of the framers of this government, every one of which was based on the immutable principle of equal rights to all.\\\"\\n—Susan B. Anthony, \\\"I Stand Before You Under Indictment\\\" (speech), 1873\\nWhich of the following was another common criticism of Andrew Carnegie?\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}