{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbed55fc-5f26-5f14-9c0d-7f5424afd6e5","task_key":"default--test--1657","task_revision_id":"1","upstream_id":"1657","short_description":"A uniformed United States Park Policeman pulled over a motorist for going…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The U.S. Park Police officer was within his rights to make an arrest, but he had to immediately transfer the defendant to state police.', 'The officer was within his authority to make the arrest, but only because the defendant acted in a disorderly manner and failed to obey an officer.', 'Federal officers, including the U.S. Park Police, cannot make arrests for state law violations under any circumstances.', \\\"The officer's authority to make an arrest depends on the specific agreement between the federal park system and the state of New York.\\\", \\\"The federal officer is allowed to arrest for homicide and serious felonies but cannot make traffic arrests within a state's geographical jurisdiction.\\\", 'The federal officer was way outside his scope of authority and he clearly cannot make an arrest outside of the confines of the federal park systems.', \\\"All federal officers can make arrests for all criminal offenses within a state's geographical jurisdiction under the priority status of federal law and federal law officers.\\\", 'The U.S. Park Police have the authority to make a warrantless arrest for a state law violation because of statutory grants of authority from the various states.', 'All U.S. Park Police officers have nationwide jurisdiction and can make arrests for any offenses, regardless of whether they occur on federal or state land.']\",\"question\":\"A uniformed United States Park Policeman pulled over a motorist for going through a red light. The man reacted gruffly, telling the officer that he had no authority to stop him. There was a brief scuffle, and the officer issued two more charges, for disorderly conduct and failure to obey an officer. Eventually, the defendant pled guilty to the red light citation and the other two charges were dismissed. He then filed a federal court action against the park policeman in which he asserted a Bivens civil rights claim for false arrest and malicious prosecution. The first legal theory was that the park officer lacked authority to stop the plaintiff on a New York street and arrest him for a state traffic violation. He claimed that the officer was outside of his federal park land jurisdiction. Will the court likely uphold the authority of the park police under these facts?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}