{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7edef513-49a7-53d5-a12d-a917e9db8dc3","task_key":"all--test--7edef513-49a7-53d5-a12d-a917e9db8dc3","task_revision_id":"1","upstream_id":"","short_description":"Police officers had probable cause to believe that drug dealing was routinely…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because exigent circumstances justified the officers' entry.\\\", \\\"No, because the motel manager consented to the officers' entry.\\\", 'Yes, because exigent circumstances cannot excuse the lack of a warrant.', 'Yes, because the officers cannot benefit from exigent circumstances that they created.']\",\"question\":\"Police officers had probable cause to believe that drug dealing was routinely taking place in a particular room at a local motel. The motel manager authorized the officers to enter the room and provided them with a passkey. Without obtaining a warrant, the officers knocked on the room's door, announced their presence, and told the occupants that they would like to speak with them. The officers then heard yelling and repeated flushing of the toilet. They then used the passkey and entered the room, where they saw the occupants dumping drugs into the toilet. The occupants of the room were charged with drug dealing and have moved to suppress the drugs. Should the court grant the motion to suppress?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}