{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9f8ca28-a589-5eac-9a7f-56a11b36f927","task_key":"all--test--f9f8ca28-a589-5eac-9a7f-56a11b36f927","task_revision_id":"1","upstream_id":"","short_description":"At a defendantÕs trial for extortion, the prosecutor called a witness expecting…","config":"all","split":"test","body":"{\"choices\":\"['Admit the testimony, because it contains a statement by a party-opponent.', 'Admit the testimony, both for impeachment and for substantive use, because the witness made the inconsistent statement under oath at a formal proceeding.', 'Admit the testimony under the former testimony exception to the hearsay rule.', 'Exclude the testimony for substantive use, because it is a testimonial statement.']\",\"question\":\"At a defendantÕs trial for extortion, the prosecutor called a witness expecting her to testify that she had heard the defendant threaten a man with physical harm unless the man made payoffs to the defendant. The witness denied ever having heard the defendant make such threats, even though she had testified to that effect before the grand jury. The prosecutor now seeks to admit the witnessÕs grand jury testimony. How should the court rule with regard to the grand jury testimony?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}