{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a410d79c-9740-5821-b736-56c9601adf1c","task_key":"all--test--a410d79c-9740-5821-b736-56c9601adf1c","task_revision_id":"1","upstream_id":"","short_description":"A defendant was prosecuted for assault and battery after he admitted striking…","config":"all","split":"test","body":"{\"choices\":\"['improper, because it is not relevant to the issues in the case.', 'improper, because the defendant has not been convicted of any crime in connection with the tax return.', \\\"within the court's discretion to allow, because filing a false income tax return is an act that bears on the witness's truthfulness.\\\", \\\"within the court's discretion to allow, because federal income tax evasion is a crime punishable by imprisonment in excess of one year.\\\"]\",\"question\":\"A defendant was prosecuted for assault and battery after he admitted striking the victim with a pool cue during a barroom argument. The defendant claimed that he acted in self-defense after he was attacked by the victim, who was drunk and belligerent. At trial, the defendant called a witness who testified that the victim was the aggressor in the altercation. On cross-examination of the witness, the prosecuting attorney asked the witness the following question:\\\"Isn't it true that when you filed your federal income tax return last year, you failed to report the interest income from your savings accounts?\\\" The prosecuting attorney was informed of this fact by the witness's accountant. Upon objection by the defendant's attorney, the prosecutor's question is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}