{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1aef15c3-83ca-58a2-ad23-4c8787514217","task_key":"all--test--1aef15c3-83ca-58a2-ad23-4c8787514217","task_revision_id":"1","upstream_id":"","short_description":"Mike is arrested on suspicion of larceny. After receiving and acknowledging his…","config":"all","split":"test","body":"{\"choices\":\"['Upheld, because he confessed with full understanding of his right to remain silent, and that his statements could be used against him.', 'Upheld, because his conviction would probably have been secured regardless of the admittance of his confession, and the decision was harmless error.', 'Upheld, because the confession may be admitted to impeach.', 'Reversed, because a coerced confession is never admissible.']\",\"question\":\"Mike is arrested on suspicion of larceny. After receiving and acknowledging his Miranda rights, he is questioned by Officer. When he refuses to confess, Officer tells Mike that he has connections in the local prison that could make Mike's life there \\\"unpleasant\\\" if he does not confess. Mike confesses. At trial, the confession is not used in the prosecution's case-in-chief, but is admitted to impeach Mike's testimony. On appeal, Mike's conviction will likely be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}