{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d5eb3f9-963a-59bf-afac-2f4cb29a603d","task_key":"default--test--1047","task_revision_id":"1","upstream_id":"1047","short_description":"An avowed corrimunist was elected vice president of a union. A senator, in his…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The statutory provision is a violation of the person's Eighth Amendment right against cruel and unusual punishment.\\\", \\\"The statutory provision is a violation of the person's Sixth Amendment right to a fair trial.\\\", 'The statutory provision in the act is a form of legislative punishment violative of the Constitution as a bill of attainder.', 'Making it a crime for a Communist to hold a union office is a suspect classification, which violates the equal protection clause.', 'The statutory prohibition is a reasonable method of discrimination since the benefit to the public outweighs the injury or restrictions that would be inflicted upon the person.', 'The statute is valid as it is a reasonable restriction on the First Amendment rights of an individual in the interest of national security.', \\\"The statutory provision is unconstitutional as it violates the person's right to due process under the Fifth Amendment.\\\", \\\"The statutory provision is a form of unconstitutional prior restraint on a person's First Amendment right of free association.\\\", \\\"The statutory prohibition is a valid exercise of Congress' power to regulate interstate commerce.\\\", \\\"The statute is unconstitutional because it infringes upon the individual's right to political affiliation, protected under the First Amendment.\\\"]\",\"question\":\"An avowed corrimunist was elected vice president of a union. A senator, in his investigation of communist infiltration of national labor unions, found a provision in a statute passed by Congress, which makes it a crime for a member of the communist party to act as an official of a labor union. After a subsequent legislative hearing, the communist is dismissed from his position by the union. Which of the following most accurately summarizes the applicable rule of constitutional law regarding the aforementioned provision of the statute?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}