{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87edf35f-81de-5c60-bac7-26437076ef7e","task_key":"all--test--87edf35f-81de-5c60-bac7-26437076ef7e","task_revision_id":"1","upstream_id":"","short_description":"A defendant who is an accountant has been charged with fraud for allegedly…","config":"all","split":"test","body":"{\"choices\":\"['No, because it is impermissible character evidence.', 'No, because it is not relevant to the issues in this case.', 'Yes, to show absence of mistake.', \\\"Yes, to show the defendant's propensity to commit the crime.\\\"]\",\"question\":\"A defendant who is an accountant has been charged with fraud for allegedly helping a client file false income tax returns by shifting substantial medical expenses from one year to another. The defendant has pleaded not guilty, claiming that he made an honest mistake as to the date the expenses were paid. At trial, the prosecutor offers evidence of the defendant's involvement in an earlier scheme to help a different client falsify tax returns in the same way. Is the evidence of the defendant's involvement in the earlier scheme admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}