{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"73434aaf-5d4f-5857-bbb6-21533598016b","task_key":"all--test--73434aaf-5d4f-5857-bbb6-21533598016b","task_revision_id":"1","upstream_id":"","short_description":"A state's constitution reserves to the people of each municipality in the state…","config":"all","split":"test","body":"{\"choices\":\"[\\\"unconstitutional, as violation of the real estate developer's due process rights.\\\", 'unconstitutional, as an unlawful delegation of legislative power to a regulatory body.', \\\"constitutional, as a valid exercise of the city's police power.\\\", 'constitutional, as a valid exercise of a power reserved by the people to themselves.']\",\"question\":\"A state's constitution reserves to the people of each municipality in the state the power of referendum with respect to all questions that the municipality is authorized to control by legislation. A real estate developer applied for a zoning change to permit construction of a convalescent home on land he owned in a city within the state. While the application was pending, the city charter was amended by popular vote so as to require that any changes in land use agreed to by the city council be approved by a 55% vote in a referendum. The city planning commission and the city council both approved the proposed zoning change. However, the real estate developer's subsequent application for approval for a proposed home on the grounds was rejected because the rezoning action had not been submitted to a referendum. In an action brought in state court seeking a judgment declaring the city charter amendment invalid, the court will most likely declare the amendment\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}