{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97accb16-7fa8-52aa-b138-8e9a7d1b95d5","task_key":"default--test--1005","task_revision_id":"1","upstream_id":"1005","short_description":"The police interrogated a suspect regarding a series of burglaries. The suspect…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The suspect's past familiarity with the procedure proves he understood the situation and his confession is valid.\\\", \\\"The suspect's signature on the waiver of Miranda rights is not enough to prove that he fully understood the situation.\\\", 'The signing of a written waiver of Miranda rights is always proof that the suspect gave a voluntary confession.', 'The outward signs all indicate that the suspect understood the procedure and gave a voluntary confession despite his disabilities.', \\\"There are too many questionable deficiencies in the suspect's ability to understand what was going on and the confession must be suppressed.\\\", 'The suspect was provided with basic needs during the interrogation, proving that it was not coercive.', \\\"The confession must be suppressed due to the suspect's intellectual disabilities and possible influence of drugs.\\\", 'He was too confused to understand what was going on and the confession was coerced by the persistence of the police.', \\\"The suspect's disabilities make it impossible for him to understand the situation, thus the confession must be suppressed.\\\", \\\"The suspect's comprehension of the Miranda rights is unclear, thus the confession must be reevaluated.\\\"]\",\"question\":\"The police interrogated a suspect regarding a series of burglaries. The suspect had attention deficit disorder, dyslexia and an IQ of 72. Prior to beginning they read a list of Miranda rights to him on a form that the suspect had in front of him. After the reading, he initialed each paragraph indicating he understood. At the bottom of the form was an express written waiver of the Miranda rights, which he signed. He had been interrogated in the past and was familiar with the procedure. During the four hours of interrogation he was given bathroom breaks, coffee, and a sandwich. At the end of the process, he made a logical and seemingly cogent confession to all of the burglaries in a taped statement. The defendant later moved to suppress the confession, arguing that he was coming down from drugs at the time and was intellectually unable to understand and comprehend the nature of the warnings or the waiver. What is the most likely decision of the court under the totality of the circumstances?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}