{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04085ae6-53d1-5e5c-902a-41cbc147c0d7","task_key":"all--test--04085ae6-53d1-5e5c-902a-41cbc147c0d7","task_revision_id":"1","upstream_id":"","short_description":"An accounting professor has been called as a defense expert witness in a…","config":"all","split":"test","body":"{\"choices\":\"['overrule the objection and permit the entire treatise to be introduced into evidence.', 'overrule the objection and permit only the contradictory statements to be read into evidence.', 'sustain the objection, because the statements in the treatise are hearsay not within any recognized exception.', 'sustain the objection, because the professor did not rely on the treatise in making his conclusions.']\",\"question\":\"An accounting professor has been called as a defense expert witness in a white-collar criminal case. He testified that accounting records conclusively demonstrated that the defendant had not committed any type of fraud. On cross-examination of the professor, the prosecuting attorney asked him if he was familiar with a particular treatise. The professor responded affirmatively. The attorney then asked him if the treatise was accepted as authoritative and if he was aware of the conclusions regarding the ability of accountants to detect fraud. The professor responded in the affirmative. The prosecutor now attempts to read the following section of the treatise into evidence: \\\"It is virtually impossible to state with conclusive certainty the existence, or lack thereof, of fraud in the majority of white-collar crime cases, given the sophistication of defendants and the variety of accounting methods available. \\\"On objection by the defendant's attorney, the trial judge will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}