{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30c58f9f-4736-5ffc-abe6-295560fe0855","task_key":"all--test--30c58f9f-4736-5ffc-abe6-295560fe0855","task_revision_id":"1","upstream_id":"","short_description":"A partnership purchased five acres of unimproved land for purposes of…","config":"all","split":"test","body":"{\"choices\":\"['This was illegal spot zoning that was unconstitutional.', 'This was an unlawful taking without prior notice and therefore unconstitutional.', 'The ordinance allows for up to five residences and therefore it is constitutional.', 'The law was a violation of the ex post facto clause of the U.S. Constitution.']\",\"question\":\"A partnership purchased five acres of unimproved land for purposes of residential development. A state law was subsequently passed requiring the city to prepare a general plan governing both land use and open-space development. The city's plan, with new density requirements, effectively limited development of the five-acre lot to a total of one to five single family residences. The partnership sued the city, asking for a declaratory judgment that the city had taken its property without just compensation in violation of the Fifth and Fourteenth Amendments. The partnership lost at the local and appellate levels. The United States Supreme Court agreed to hear the case €” what did it decide?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}