{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79629656-c290-5f10-9ea2-2ec06eec5cf7","task_key":"all--test--79629656-c290-5f10-9ea2-2ec06eec5cf7","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['Homestead Act of 1862.', 'Immigration Act of 1921.', 'Supreme Court decision in Plessy v. Ferguson.', 'Dawes Act of 1887.']\",\"question\":\"This question refers to the following information.\\n\\\"I know that whenever the subject has occurred in conversation where I have been present, it has appeared to be the opinion of every one that we could not be taxed by a Parliament wherein we were not represented. But the payment of duties laid by an act of Parliament as regulations of commerce was never disputed. . . . An external tax is a duty laid on commodities imported; that duty is added to the first cost and other charges on the commodity, and, when it is offered for sale, makes a part of the price. If the people do not like it at that price, they refuse it; they are not obliged to pay it. But an internal tax is forced from the people without their consent if not laid by their own representatives. The Stamp Act says we shall have no commerce, make no exchange of property with each other, neither purchase nor grant, nor recover debts; we shall neither marry nor make our wills, unless we pay such and such sums; and thus it is intended to extort our money from us or ruin us by the consequence of refusing to pay it.\\\"\\n—Benjamin Franklin, Examination before Parliament, 1766\\nPresident Jackson's policy was later altered by the\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}