{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f4cef1a-3c0a-5fc9-9dbe-49f254dc1753","task_key":"all--test--0f4cef1a-3c0a-5fc9-9dbe-49f254dc1753","task_revision_id":"1","upstream_id":"","short_description":"A company operates passenger buses to all the major cities on the east coast.…","config":"all","split":"test","body":"{\"choices\":\"[\\\"constitutional, because it is within the city's police power to regulate transportation services in the center city business district.\\\", 'constitutional, because it is a valid exercise of municipal regulation in the area of intrastate commerce.', 'unconstitutional, because it violates the dormant commerce clause.', 'unconstitutional, because the ordinance is discriminatory per Se.']\",\"question\":\"A company operates passenger buses to all the major cities on the east coast. This service is authorized under a certificate of convenience and necessity issued by the Interstate Commerce Commission, pursuant to federal statute. The certificate does not, however, specify particular highways, streets, or locations for the bus service. The company's advertising stresses that it picks up and delivers passengers at the center of each city that it serves. The company's management regards this as a particularly effective advertising point in competition with the airlines and the railroads, because short-haul traffic supplies a major part of the bus company's revenues. One of the major cities on the east coast, acting to relieve traffic congestion and air pollution, has recently enacted an ordinance that prohibits (a) the operation of all trucks and buses in a five- square- mile central business area, known as center city, between the hours of 10:00 A. M. and 4:00 P. M. on weekdays, and (b) all on-street parking of passenger automobiles in center city between the same weekday hours. The company's bus station in this city is located in the heart of the center city area. According to its transportation schedules, more than 75 buses either enter or leave this station between the hours of 10:00 A. M. and 4:00 P. M. each weekday. If the company brings suit challenging the constitutionality of the city ordinance, the court will most likely declare the ordinance\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}