{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11b5d940-dc2c-508f-92a5-a0d88477eb39","task_key":"all--test--11b5d940-dc2c-508f-92a5-a0d88477eb39","task_revision_id":"1","upstream_id":"","short_description":"A defendant was prosecuted for aggravated assault after he stabbed a man in the…","config":"all","split":"test","body":"{\"choices\":\"[\\\"proper, because it tends to show the witness's knowledge of the defendant's reputation.\\\", \\\"proper, because it is relevant to prove the defendant's violent propensities.\\\", 'not proper, because it puts into evidence separate and unrelated offenses.', 'not proper, because character cannot be proved by generalities.']\",\"question\":\"A defendant was prosecuted for aggravated assault after he stabbed a man in the chest during a dispute over a gambling debt. The defendant claimed he acted in self-defense after the man drew a knife on him and made menacing gestures with it. The defendant called a witness to the stand to testify that the defendant had a reputation in the community for being a peaceful person. On cross-examination of the witness, the prosecuting attorney asked the witness if he had heard that the defendant had often engaged in fights and brawls. Upon objection by the defendant's attorney, the trial judge should rule the question\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}