{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e34c6939-1fce-57b2-aa6d-4a93b34b0bdc","task_key":"all--test--e34c6939-1fce-57b2-aa6d-4a93b34b0bdc","task_revision_id":"1","upstream_id":"","short_description":"A state built a racetrack that was specially designed and constructed for…","config":"all","split":"test","body":"{\"choices\":\"[\\\"constitutional, because it is a justifiable exercise of the state's police power.\\\", 'unconstitutional, because it denies citizens under the age of 65 the equal protection of the law.', \\\"unconstitutional, because it impairs the obligation of the bondholders' contract.\\\", 'unconstitutional, because it is an ex post facto law.']\",\"question\":\"A state built a racetrack that was specially designed and constructed for thoroughbred horseracing. State bonds were issued to finance the construction of the racetrack. The bond agreement provided that for the first five years the racetrack was in operation, at least $2 from each admission charge would be used to repay the bond debt. The bond agreement further stipulated that if the proceeds from the admission charges during the first five years were not sufficient to repay the bondholders, then at least $1 from each admission charge for the next five years would be applied to make the necessary bond payments. After the racetrack was built and in operation for two years, the state legislature passed a bill requiring the racetrack to admit all senior citizens over the age of 65 for the discounted admission fee of 50 cents. This law is probably\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}