{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c293981-3bd4-5f2a-b93d-9fffa2276cdb","task_key":"all--test--7c293981-3bd4-5f2a-b93d-9fffa2276cdb","task_revision_id":"1","upstream_id":"","short_description":"Voters in a city passed a referendum legalizing gambling in a certain section of…","config":"all","split":"test","body":"{\"choices\":\"['unconstitutional, because the appointment of a clergy member to the commission fosters excessive governmental entanglement with religion.', 'unconstitutional on its face, because members of the commission are vested with enforcement powers.', 'constitutional, because the primary effect of appointing only one religious member to the commission does not, per Se, advance or inhibit religion.', 'constitutional, because commission membership is an appointive privilege and not an elective right.']\",\"question\":\"Voters in a city passed a referendum legalizing gambling in a certain section of the city. The law established a commission, consisting of five individuals, that was in charge of licensing and overseeing the activities of the casinos. The members of the commission were to be appointed by the mayor. Pursuant to his statutory power, the mayor appointed four private citizens and a clergy member to the Commission. There is a constitutional challenge to the appointment of the clergy member as violating the establishment clause of the First Amendment. The mayor's action is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}