{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45ac4080-23c3-5781-a5da-d5e6f6a46b67","task_key":"all--test--45ac4080-23c3-5781-a5da-d5e6f6a46b67","task_revision_id":"1","upstream_id":"","short_description":"Before a proposal to merge two towns was placed on a ballot, the state attorney…","config":"all","split":"test","body":"{\"choices\":\"['not hear the case, because it was decided on independent state grounds.', 'not hear the case, but may have it remanded to federal district court.', 'hear the federal issues involved, but decline to rule on the state issue.', 'rely on the advisory opinion rendered by the state attorney general and not hear the case on its merits.']\",\"question\":\"Before a proposal to merge two towns was placed on a ballot, the state attorney general issued an advisory opinion stating that the measure did not, in her opinion, violate any statutory or constitutional provisions. Thereafter, the proposal was placed on the ballot and was overwhelmingly passed by the voters in both towns. After the election, but before the merger had officially been carried out, two taxpayers from one of the towns initiated suit to enjoin the unification, attacking the constitutionality of the proposal. The suit reached the state supreme court and was found to be constitutional under both state and federal law. The plaintiffs now file a motion seeking to have thiscase reviewed by the U. S. Supreme Court. The Court may\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}