{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17377ae8-5682-54bf-9b16-c89633f83be2","task_key":"default--test--1751","task_revision_id":"1","upstream_id":"1751","short_description":"A state enacts a statute that will allocate funds generated by state taxes to…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"unconstitutional, as violative of the First Amendment's establishment clause.\\\", \\\"constitutional, as nonviolative of the First Amendment's establishment clause.\\\", \\\"constitutional, as nonviolative of the Fifth Amendment's due process clause.\\\", \\\"constitutional, as nonviolative of the First Amendment's free exercise clause.\\\", \\\"unconstitutional, as violative of the Fourteenth Amendment's equal protection clause.\\\", 'constitutional, as being within the area of compelling state interest.', 'constitutional, as being within the area of legitimate state interest.', \\\"unconstitutional, as violative of the First Amendment's free exercise clause.\\\", \\\"unconstitutional, as violative of the Fifth Amendment's due process clause.\\\", \\\"unconstitutional, as violative of the First Amendment's establishment clause as applicable through the Fourteenth Amendment.\\\"]\",\"question\":\"A state enacts a statute that will allocate funds generated by state taxes to pay for one-half of the annual salaries of teachers in private schools, whether the schools are public or private, as long as the schools satisfy the requirements established by the United States Department of Education. In an action brought by several state taxpayers, all avowed atheists, to challenge the constitutionality of the statute, the enactment will most likely be declared\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}