{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75b5822a-0a82-5d68-89f8-6b95a0827e43","task_key":"default--test--1328","task_revision_id":"1","upstream_id":"1328","short_description":"Police officers had probable cause to believe that a man had committed a series…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, because the officers discovered the cocaine during a lawful protective sweep of the house looking for the man's accomplice.\\\", 'Yes, because the man was not given a chance to surrender before the officers entered the house.', 'Yes, because the officers did not find the accomplice in the house.', 'No, because the officers had a valid arrest warrant for the man.', 'Yes, because the officers entered the house without permission.', 'Yes, because the officers did not have probable cause to believe that cocaine would be in the closet.', \\\"No, because the search was incident to the man's arrest.\\\", 'No, because the officers had probable cause to believe that the man was involved in criminal activity.', 'No, because the cocaine was in plain view when the officers opened the closet.', 'Yes, because the officers did not have a search warrant.']\",\"question\":\"Police officers had probable cause to believe that a man had committed a series of armed bank robberies with an accomplice. The officers obtained a valid arrest warrant for the man and went to his house to arrest him. The officers found the front door ajar and could hear a radio playing inside. They knocked on the door, identified themselves as police officers, and stated that they had a warrant. Getting no response, the officers entered the house. After the officers called the man's name several times and again identified themselves as police officers, the man emerged from the basement and was arrested. The officers went into the basement to look for the accomplice. They opened a closet door and found cocaine in plain view. They did not find the accomplice. The man was indicted for cocaine possession. He has moved to suppress the cocaine as evidence on the ground that it was obtained in violation of the Fourth Amendment. Should the court grant the motion to suppress?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}