{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83a5c72e-e2da-50cd-a987-c193d7fd69f8","task_key":"all--test--83a5c72e-e2da-50cd-a987-c193d7fd69f8","task_revision_id":"1","upstream_id":"","short_description":"A husband and his wife are involved in a contested divorce and child custody…","config":"all","split":"test","body":"{\"choices\":\"['admitted, because the spousal privilege is inapplicable, since the parties are living in separation.', 'admitted, because the marital communication privilege is inapplicable in a lawsuit involving the parties.', 'excluded, because the wife is incompetent to testify to marital communications because she is not the holder of the privilege.', 'excluded, because confidential statements made during a joint consultation are privileged communications between clients and their attorneys.']\",\"question\":\"A husband and his wife are involved in a contested divorce and child custody battle. Ignorant of the adversarial system, they both visited a family law attorney together for an initial consultation. The attorney advised them that he could not represent them both. The wife found another attorney. During the trial, the wife's attorney calls the wife to testify. She states that during the initial consultation she and her husband had with his attorney, she privately told the attorney, when her husband was taking a bathroom break, that her husband had a bad drinking problem, which was one of her major reasons for seeking a divorce and custody of the children. Upon objection by the husband's attorney, the wife's testimony should be\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}