{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf5a3bca-5d9d-5f17-9eab-893c5a72e600","task_key":"all--test--cf5a3bca-5d9d-5f17-9eab-893c5a72e600","task_revision_id":"1","upstream_id":"","short_description":"A defendant was suspected of having burglarized his neighbor's apartment. The…","config":"all","split":"test","body":"{\"choices\":\"['admit the confession because it was voluntary, and the clothing because it was obtained pursuant to a valid search warrant.', 'suppress the confession because it was obtained in violation of his Sixth Amendment right to counsel, but admit the clothing because it was obtained pursuant to a valid search warrant.', 'suppress the confession because the defendant failed to sign the waiver, but admit the clothing because it was obtained pursuant to a valid search warrant.', 'suppress the confession because it was obtained in violation of his Fifth Amendment right to counsel, and the clothing because the search warrant was secured as a result of the confession.']\",\"question\":\"A defendant was suspected of having burglarized his neighbor's apartment. The neighbor reported that his apartment had been ransacked and several items of clothing had been stolen. During the course of their investigation, two police detectives went to the defendant's place of work to interview him. After being advised of his Miranda rights, the defendant requested permission to call his attorney. Although his attorney was unavailable, the attorney's receptionist admonished him not to say anything. The defendant told the detectives he would have nothing further to say unless his attorney was present. The detectives then asked him if he would accompany them to the police station to answer some questions about an unrelated robbery. The defendant agreed. As they were driving to the police station, the defendant was asked where he had purchased the boots that he was wearing. He refused to answer. When they arrived at the police station, the detectives requested that he sign a waiver of his right to counsel. The defendant replied that he would not sign anything in the absence of his attorney. He was then placed in an interrogation room. Shortly thereafter, the defendant was told he could leave if he would be willing to say where he had purchased his boots. The defendant admitted that he had bought the boots and some shirts from a friend. From this information, the detectives obtained a search warrant and went to the defendant's home where they found the stolen clothing. The defendant is charged with burglarizing the apartment. At trial, the court should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}