{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"92f58df4-b72b-51e9-b5bf-36fb2d8b5a2b","task_key":"all--test--92f58df4-b72b-51e9-b5bf-36fb2d8b5a2b","task_revision_id":"1","upstream_id":"","short_description":"A man was under custodial interrogation as a suspect in the murder of a child.…","config":"all","split":"test","body":"{\"choices\":\"['No, the circumstances indicate that he understood that he could stop the questioning and get a lawyer but he did not do so.', \\\"No, whenever a written list of Miranda warnings is given to a suspect and he doesn't ask for a lawyer, this is proof that the suspect fully understands his rights.\\\", 'Yes, the confession must be suppressed because he was questioned for over three hours, making anything he said essentially involuntary.', 'Yes, the use of religious deception to get a suspect to admit to crime renders the confession involuntary.']\",\"question\":\"A man was under custodial interrogation as a suspect in the murder of a child. The police initially gave him a written list of his Miranda rights. They told him to read the list out loud, which he did. They then asked him if he understood the right to remain silent and to have a lawyer present and he nodded affirmatively. He refused to sign the writing but did not ask for an attorney. During the next three hours they interrogated him and got few answers; he was largely silent but did respond at times with a \\\"no\\\" or \\\"yes\\\" or a nod of the head. Then one of the detectives asked him if he was ready to ask God for forgiveness for killing the child. His eyes welled with tears and he said \\\"yes.\\\" This opened further questioning leading to his admission of guilt. He would not sign a written confession or a waiver of the right to remain silent. He was arrested for murder and moved to suppress the confession. Will the court suppress the confession as being involuntarily given?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}