{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ca6b662-6a55-52e4-9bec-a129cf20405c","task_key":"all--test--9ca6b662-6a55-52e4-9bec-a129cf20405c","task_revision_id":"1","upstream_id":"","short_description":"A defendant was being prosecuted on a charge of manslaughter in the first degree…","config":"all","split":"test","body":"{\"choices\":\"[\\\"objectionable, because the defendant's attorney did not lay a proper foundation for impeaching his own witness.\\\", 'objectionable, because a party may not impeach his own witness.', \\\"unobjectionable, because a witness's testimony before a grand jury is more reliable because it occurred shortly after the abortion was allegedly performed.\\\", 'unobjectionable, because a party can impeach his own witness by a prior inconsistent statement.']\",\"question\":\"A defendant was being prosecuted on a charge of manslaughter in the first degree for causing the death of a 15-year-old girl on whom he had allegedly performed an abortion in his private clinic. A witness for the defendant testified to the defendant's non-culpability at the grand jury proceeding. The witness was later called as a defense witness at the trial. During his direct examination, defendant's attorney asked him, \\\"Is it not true that the defendant could not have performed the abortion on the decedent since he was assisting you in another abortion at the time?\\\" The witness answers in the negative and states that the defendant did, in fact, perform the abortion on the girl. The defendant's attorney then assails the witness's testimony as a recent fabrication, and asks the witness if he had not stated to a grand jury that the defendant was not guilty of performing the abortion, thus causing the death of the girl. The prosecutor objects to this question. The trial judge should rule that this question is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}