{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2a37e2c5-e65b-510a-ba2a-3c6b4ecec588","task_key":"all--test--2a37e2c5-e65b-510a-ba2a-3c6b4ecec588","task_revision_id":"1","upstream_id":"","short_description":"A president of the senior class at a public high school, on his own initiative…","config":"all","split":"test","body":"{\"choices\":\"['grant relief, since the invocation violates the establishment clause of the First Amendment.', 'grant relief, since the primary effect of the invocation is to advance religious beliefs.', 'deny relief, since the lunch hour is not part of the educational process.', 'deny relief, since the noncompulsory nature of the invocation would not be violative of the establishment clause.']\",\"question\":\"A president of the senior class at a public high school, on his own initiative instituted among the students the practice of invoking divine blessing at the start of the daily luncheon served in the high school cafeteria. Although no teacher or school official either encouraged or discouraged the practice, all but two teachers, when present, joined in the invocation. A student and his parents protested to the school superintendent, but they refused to intervene on the grounds that the matter was entirely up to the students. Moreover, school board officials pointed out that the students who led the recital always sat at a table in the rear of the cafeteria and that no one was required to participate in the blessing. In an action by the student's parents to enjoin the daily luncheon invocation at the high school, the court will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}