{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7d1196a-375c-5a72-b4ac-efb281336bdc","task_key":"default--test--1565","task_revision_id":"1","upstream_id":"1565","short_description":"A state statute contained reasonable provisions for the regulation, sale,…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The statute is a reasonable regulation that will not hinder the right to possess a handgun for self-defense in the home and the individual's lawsuit will be dismissed.\\\", \\\"The statute is unconstitutional because it violates the Second Amendment, thus the individual's lawsuit will be successful.\\\", \\\"The statute is invalid as it excessively regulates the possession and use of firearms, thus the individual's lawsuit will be successful.\\\", \\\"The statute is unreasonable as it infringes on the individual's right to bear arms, thus the man's lawsuit will be successful.\\\", \\\"The Second Amendment only applies to militias, thus the individual's lawsuit will be dismissed.\\\", 'Semi-automatic guns cannot be restricted in any way and the lawsuit will be successful.', \\\"The Second Amendment prohibits all forms of gun control because of the need for a strong militia, and the man's lawsuit will be successful.\\\", \\\"The statute is a reasonable regulation but the individual's lawsuit will still be successful due to a technicality in the law.\\\", \\\"The statute is constitutional as it aims to prevent the misuse of firearms, thus the individual's lawsuit will be dismissed.\\\", \\\"The regulation of firearms for the safety of society is an absolute, unfettered right of the state governments and the individual's lawsuit is dismissed for that reason.\\\"]\",\"question\":\"A state statute contained reasonable provisions for the regulation, sale, possession and use of certain weapons for self-defense in one's home, including semi-automatic rifles and handguns. The statute stressed registration and the individual's personal characteristics. The law prohibited the issuance of a license to felons and mentally ill persons or others presenting a danger of violence. It prohibited the commercial sale of guns from residential locations and required a commercial permit. Certain semi-automatic weapons were prohibited. A citizen who wished to purchase and maintain various guns in a manner violative of the statute sued the state. He claimed that the statute was a violation of the Second Amendment, which provides for the \\\"right to keep and bear arms.\\\" The case was accepted for review by the U.S. Supreme Court. What will the Court decide?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}