{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2950c81-5c06-5561-b1b2-dfbb4d5c57b2","task_key":"default--test--1615","task_revision_id":"1","upstream_id":"1615","short_description":"A state imposes a graduated income tax upon net income calculated under federal…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['uphold the action, because the state law discriminates against public school students.', 'dismiss the action, because the suit involves a political question.', 'uphold the action, because the plaintiffs have a valid claim under the First Amendment.', 'dismiss the action, because the plaintiffs have not suffered any actual harm.', 'dismiss the action, because the plaintiffs lack standing as taxpayers.', \\\"uphold the action, because the state law interferes with the parents' right to choose a public school.\\\", 'dismiss the action, because the issues being litigated are moot.', 'dismiss the action, because the issues are not ripe.', 'dismiss the action, because the lawsuit is beyond the jurisdiction of the federal court.', 'dismiss the action, because the state law does not violate the establishment clause of the First Amendment.']\",\"question\":\"A state imposes a graduated income tax upon net income calculated under federal law. Section 22 of the city's reform act of 2010, which is to become effective on January 1, 2010, provides: \\\"Any parent or guardian financially responsible for the education of his ward may claim a direct tax credit against his income tax liability equal to the amount of tuition of a child or children of high school age who does not attend a public high school. \\\" Other provisions define \\\"tuition\\\" very broadly but limit the credit to tuition paid to schools meeting the educational requirements as determined by the state. On December 1, 2009, the parents of two students who attend a public high school in the state sue for a declaratory judgment and injunction in federal court, claiming that Section 22 violates the establishment clause of the First Amendment. The federal court will most likely\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}