{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6ce266f5-1695-5c85-822c-2271a9a1e588","task_key":"all--test--6ce266f5-1695-5c85-822c-2271a9a1e588","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff was the varsity basketball coach of the boys' team at a high school.…","config":"all","split":"test","body":"{\"choices\":\"['admissible, for impeachment purposes only.', 'admissible, as substantive evidence.', 'inadmissible, because specific instances of conduct is an improper method of proving character.', 'inadmissible to prove character, but admissible for the purpose of showing intent or common scheme.']\",\"question\":\"A plaintiff was the varsity basketball coach of the boys' team at a high school. After the second game of the season, the plaintiff was suddenly dismissed as the coach for undisclosed reasons. After the dismissal was made public, the local newspaper ran an article discussing the plaintiff's tenure as coach. The article, which was written by the sports editor, stated that \\\"The plaintiff was a bad role model and morally unfit for the coaching job. \\\"The plaintiff brought suit against the newspaper for defamation. At trial, the plaintiff called a player on the basketball team to testify that \\\"the plaintiff was a good role model and respected by the players. \\\" In presenting its evidence, the newspaper called a witness to testify. He proposes to testify that he saw the plaintiff give beer to players on three occasions after victories. Upon objection by the plaintiff's attorney, witness's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}