{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13015a2c-c201-5fbf-a4be-b8f0e610cff6","task_key":"all--test--13015a2c-c201-5fbf-a4be-b8f0e610cff6","task_revision_id":"1","upstream_id":"","short_description":"A defendant has been charged with making a false statement to a federally…","config":"all","split":"test","body":"{\"choices\":\"['No, because even though the wife is a willing witness, the defendant has the right to exclude confidential marital communications in federal court.', 'No, but only if the law of the state where the defendant and his wife reside recognizes a privilege for confidential marital communications.', 'Yes, because in federal court the right not to testify belongs to the testifying spouse, and she is a willing witness.', 'Yes, because while the adverse testimonial privilege is recognized in federal court, the marital communications privilege is not.']\",\"question\":\"A defendant has been charged with making a false statement to a federally insured financial institution to secure a loan. At trial, the prosecutor calls the defendant's wife as a willing witness to testify that the defendant told her in confidence that he had misrepresented his assets on the loan application. The defendant objects to his wife's testimony. Should the testimony be admitted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}