{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2062535-4d8b-5f5c-af09-ae2925bbf2f4","task_key":"all--test--b2062535-4d8b-5f5c-af09-ae2925bbf2f4","task_revision_id":"1","upstream_id":"","short_description":"A defendant was arrested and charged with forgery for unlawfully signing her…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible as evidence of the defendant's propensity for committing the crime.\\\", 'admissible as an admission.', 'inadmissible, because it was written during the course of plea bargain negotiations.', 'inadmissible, because specific instances of misconduct are not admissible in a criminal case.']\",\"question\":\"A defendant was arrested and charged with forgery for unlawfully signing her ex-husband's signature on his pay check. Before trial, the defendant entered into plea bargain negotiations with the prosecution. During the course of these negotiations, the defendant sent a letter to the prosecuting attorney in which she stated: \\\"I don't want to go to jail, so if you recommend a suspended sentence, I'll plead guilty and admit that I forged my ex-husband's signature on the check. \\\" Subsequently, the plea bargain negotiations fell through. Shortly thereafter, the defendant was arrested and charged with forging her ex-husband's signature on his next pay check. She has denied culpability regarding the second forgery charge. At trial for the alleged second forgery, the prosecution seeks to introduce the letter the defendant sent to the prosecution during her earlier plea bargain negotiations. The defendant's attorney objects to the admissibility of the letter. The trial court judge should rule the letter is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}