{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ee345b3-f525-550a-9eb8-e9e2ec74aef0","task_key":"all--test--2ee345b3-f525-550a-9eb8-e9e2ec74aef0","task_revision_id":"1","upstream_id":"","short_description":"A state in which several popular ski resorts are located has enacted a law…","config":"all","split":"test","body":"{\"choices\":\"['No, because her ability to contract for employment with the state is not being impaired.', 'No, because she has no existing contract that is being impaired.', 'Yes, because, as applied to an otherwise qualified ski instructor, the requirement is not rationally related to a legitimate government interest.', 'Yes, because the requirement substantially impairs her ability to contract for employment in the absence of an important government objective.']\",\"question\":\"A state in which several popular ski resorts are located has enacted a law requiring state certification of ski instructors. Under the law, applicants for certification must attend a monthlong course, pass a test, and pay a fee of $1,000. The stated purpose of the law is to \\\"promote uniformity in the methods of ski instruction at the state's ski areas.\\\" Shortly before the law's enactment, when the state did not require certification of ski instructors, a woman moved to the state intending to find employment as a ski instructor. The woman had been a ski instructor in another state for many years. But since her move, even though ski resorts in the state have found her to be highly qualified, no resort will offer her a contract for employment as a ski instructor because she has not received the required state certification. As applied to the woman, does the state certification requirement constitute a violation of the contracts clause?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}