{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d472a61-4f02-525b-9be6-77def1535432","task_key":"all--test--9d472a61-4f02-525b-9be6-77def1535432","task_revision_id":"1","upstream_id":"","short_description":"A defendant was charged with killing a victim during a barroom fight. The…","config":"all","split":"test","body":"{\"choices\":\"['admissible, for the purpose of rebutting the contention that the defendant was a peaceable person.', \\\"admissible, for the purpose of repudiating the defendant's claim of self-defense.\\\", \\\"inadmissible, because the victim's character cannot be proved by the brother-in-law's testimony.\\\", \\\"inadmissible, because the defendant did not directly attack the victim's character.\\\"]\",\"question\":\"A defendant was charged with killing a victim during a barroom fight. The defendant claimed that he acted in self-defense when he was attacked by the victim. At trial, the defendant called a witness who testified that he witnessed the altercation and that the victim was the aggressor. The witness further testified that he has known the defendant for 10 years and that the defendant is a peaceable man who was acting reasonably in self-defense. The prosecution then offered the testimony of the victim's brother-in-law, who proposed to testify that the victim was a peaceable and law-abiding person. Upon objection by the defendant's attorney, the brother-in-law's proposed testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}