{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35b8298f-b5ab-5e9e-80b5-3cfb00d9f1ec","task_key":"all--test--35b8298f-b5ab-5e9e-80b5-3cfb00d9f1ec","task_revision_id":"1","upstream_id":"","short_description":"A state passed Public Health Law 2122(a), which provides for exemption from…","config":"all","split":"test","body":"{\"choices\":\"[\\\"Yes, it will be invalidated because it cannot override the state's interest in having its school students immunized.\\\", 'No, it will not be invalidated \\\\x80\\\\x93 it is a fair exemption that the state has chosen to provide for religious groups.', 'Yes, it will be invalidated because when an exemption is provided it may not unfairly discriminate and favor one type of religion or religious belief over another.', \\\"No, it won't be invalidated because it is a valid exercise of the state's right to protect the privacy of its citizens.\\\"]\",\"question\":\"A state passed Public Health Law 2122(a), which provides for exemption from immunization for school age children if they are \\\"...members of a recognized religious organization whose beliefs are opposed to immunizations...\\\" A couple applied to the school district for an exemption from immunizations on behalf of their child. They were not members of a recognized religious organization; instead, they requested an exemption based on their personal spiritual beliefs against immunization. Will the courts declare the existing exemption to be invalid?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}