{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7dcd7964-cc16-5dd1-8628-d6da6bddd864","task_key":"all--test--7dcd7964-cc16-5dd1-8628-d6da6bddd864","task_revision_id":"1","upstream_id":"","short_description":"At a defendant's trial for burglary, the defendant has called a witness who has…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the witness's character for truthfulness cannot be attacked by specific instances of conduct.\\\", \\\"No, because the witness's failure to mention the alibi is collateral and ambiguous.\\\", 'Yes, as impeachment for bias and interest.', 'Yes, as impeachment for prior inconsistency.']\",\"question\":\"At a defendant's trial for burglary, the defendant has called a witness who has testified without objection that the defendant said shortly after his arrest, \\\"They've got the wrong person for this, because I have an alibi.\\\" The prosecutor seeks to cross- examine the witness about why she did not mention that statement when the police asked her whether the defendant had said anything to her about having an alibi. Is the prosecutor's proposed cross-examination proper?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}