{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ae49f01-616f-5bdd-8ec7-2318d3f2a3b3","task_key":"all--test--3ae49f01-616f-5bdd-8ec7-2318d3f2a3b3","task_revision_id":"1","upstream_id":"","short_description":"A police department filed a complaint charging Bill with the sale of narcotic…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because it was on the person, or in the presence of the person, who was later arrested.', 'admissible, because it was obtained pursuant to a valid search.', 'inadmissible, because the letter was obtained as a result of a non-consensual search and seizure.', 'inadmissible, because the letter was the fruit of an illegal search and seizure.']\",\"question\":\"A police department filed a complaint charging Bill with the sale of narcotic drugs in violation of a state statute and obtained a valid warrant for his arrest. Two police officers went to Bill's house, a two-story colonial. After knocking at the door, they were admitted by Bill's brother, Mark. One of the policemen showed Mark the arrest warrant and asked if he was Bill. He replied that he was Bill's brother and that he was staying at his brother's house while Bill was away on a business trip. The police officer then asked Mark for some identification. Mark was only able to produce an out-of-state driver's license. The other policeman then became suspicious and proceeded to frisk Mark. During the pat-down, the police officer found the following letter, which read:\\\"Dear Mark,If you could get rid of your brother permanently, we could be together forever. With love,your favorite girl. \\\"Nervously, Mark then agreed to allow the policemen to search the house. The officers conducted an extensive two-hour search of the house and found nothing of interest to them. As they were leaving, one of the police officers walked over to a parked car in the driveway of the house and noticed that the car's trunk was slightly open. When he looked in the trunk, the police officer discovered Bill's body. If Mark's attorney files a motion to suppress the introduction of the letter into evidence, the court should rule that the evidence is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}