{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2369e80-7163-50ba-9b50-a179667606aa","task_key":"default--test--1655","task_revision_id":"1","upstream_id":"1655","short_description":"A state has a statute generally prohibiting the installation and use of radar…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['constitutional, because it can be considered a reasonable restriction on the right to privacy.', 'unconstitutional, because it unduly burdens interstate commerce.', 'unconstitutional, because the state statute fails to give credit to the law of a neighboring state.', 'constitutional, because the commerce clause does not invalidate a state regulation relating to interstate commerce unless Congress takes express action to do so.', 'constitutional, because states have the power to regulate their own roads and highways.', 'constitutional, because it protects a legitimate state interest.', 'unconstitutional, because it infringes on the rights of residents of other states.', 'unconstitutional, because the state statute is preempted by federal law.', \\\"unconstitutional, because it violates the Fourth Amendment's prohibition against unreasonable searches and seizures.\\\"]\",\"question\":\"A state has a statute generally prohibiting the installation and use of radar detection devices in any motor vehicle operating on the roadways within the state. This prohibition was enacted to prevent motorists from evading radar or speed checkpoints on county and state highways. A neighboring state has no such regulation in effect. By the same token, Congress has taken no action on the matter. A resident of the neighboring state has installed a radar detection device in his automobile. While driving to visit his mother he is arrested and charged with violating the aforementioned state statute. The resident files a complaint challenging the constitutionality of the state statute. As applied to the resident, the state prohibition against the use of radar detection devices is likely to be held\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}