{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6e08169-436d-5474-8491-76728720bafd","task_key":"all--test--f6e08169-436d-5474-8491-76728720bafd","task_revision_id":"1","upstream_id":"","short_description":"As a legislative aide to a U. S. Senator you are called upon to provide an…","config":"all","split":"test","body":"{\"choices\":\"['constitutional, because the fact that the tax applies to all sales of every type of birth control device invalidates any possible objection to the tax on the grounds that it violates the equal protection clause of the Fourteenth Amendment.', 'constitutional, because the fact that controversial policy motives may have induced the enactment of an otherwise reasonable measure calculated to raise revenue does not ipso facto invalidate the tax.', 'unconstitutional, because in inseverable aggregates, the domestic purchases and sales of birth control devices affect interstate and foreign commerce.', 'unconstitutional, because the tax burdens the fundamental right to privacy of users of birth control devices without establishing a compelling national interest for doing so.']\",\"question\":\"As a legislative aide to a U. S. Senator you are called upon to provide an analysis of the constitutionality of a bill pending congressional approval. The bill imposes a 15% tax upon the gross annual receipts from the sales of all birth control devices. The bill has the strong support of conservative and pro-life organizations. The stated purpose of the proposed measure is to raise revenue and spur population growth across the country. In your learned opinion, the proposed tax is probably\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}