{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32f91c7d-ba37-55f9-a215-4404f95bec34","task_key":"all--test--32f91c7d-ba37-55f9-a215-4404f95bec34","task_revision_id":"1","upstream_id":"","short_description":"A state enacts a statute that will allocate funds generated by state taxes to…","config":"all","split":"test","body":"{\"choices\":\"[\\\"unconstitutional, as violative of the First Amendment's establishment clause.\\\", \\\"unconstitutional, as violative of the First Amendment's establishment clause as applicable through the Fourteenth Amendment.\\\", \\\"constitutional, as nonviolative of the First Amendment's establishment clause.\\\", 'constitutional, as being within the area of compelling state interest.']\",\"question\":\"A state enacts a statute that will allocate funds generated by state taxes to pay for one-half of the annual salaries of teachers in private schools, whether the schools are public or private, as long as the schools satisfy the requirements established by the United States Department of Education. In an action brought by several state taxpayers, all avowed atheists, to challenge the constitutionality of the statute, the enactment will most likely be declared\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}