{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2ba3bd6-6b31-5f4f-8e8b-1229160f56ca","task_key":"default--test--1878","task_revision_id":"1","upstream_id":"1878","short_description":"Police officers had probable cause to believe that drug dealing was routinely…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the sound of flushing toilet provided sufficient reason for immediate entry.', \\\"Yes, because the motel manager did not have the authority to consent to the officers' entry.\\\", 'Yes, because the officers did not have a search warrant.', 'Yes, because the officers cannot benefit from exigent circumstances that they created.', \\\"No, because exigent circumstances justified the officers' entry.\\\", 'Yes, because the officers should have waited for the occupants to open the door.', 'No, because the occupants were actively destroying evidence.', 'Yes, because exigent circumstances cannot excuse the lack of a warrant.', \\\"No, because the motel manager consented to the officers' entry.\\\", 'No, because the officers had probable cause to enter the room.']\",\"question\":\"Police officers had probable cause to believe that drug dealing was routinely taking place in a particular room at a local motel. The motel manager authorized the officers to enter the room and provided them with a passkey. Without obtaining a warrant, the officers knocked on the room's door, announced their presence, and told the occupants that they would like to speak with them. The officers then heard yelling and repeated flushing of the toilet. They then used the passkey and entered the room, where they saw the occupants dumping drugs into the toilet. The occupants of the room were charged with drug dealing and have moved to suppress the drugs. Should the court grant the motion to suppress?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}