{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0762306e-5d9c-584c-8503-4d398be25e0b","task_key":"all--test--0762306e-5d9c-584c-8503-4d398be25e0b","task_revision_id":"1","upstream_id":"","short_description":"A victim and a defendant both worked as longshoremen at a shipyard. After the…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because it reports a declaration against interest.', \\\"admissible, because it reports the defendant's adoptive admission of the victim's assertion.\\\", 'inadmissible, because of the principle of collateral estoppel.', 'inadmissible, because it is hearsay not within any recognized exception.']\",\"question\":\"A victim and a defendant both worked as longshoremen at a shipyard. After the victim was shot to death, the defendant was tried for murder and acquitted. Following the acquittal, the victim's estate sued the defendant in a wrongful death action. During the civil trial, the victim's estate called a witness to testify. The witness, who worked with both men at the shipyard, testified that two weeks before the shooting, the victim came to work with a broken nose and said that the defendant had caused it. The attorney for the victim's estate then asked the witness the following question, \\\"Was the defendant present during your conversation with the victim, and if so, did he say anything about the victim's broken nose?\\\" The witness replied, \\\"Yes, the defendant was present, and after the victim told me that the defendant broke his nose, the defendant said, €˜And that's only the beginning. \\\"Upon objection by the defendant's attorney, the witness's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}