{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"91f9aa54-01ac-5ab7-a760-0517255eb533","task_key":"all--test--91f9aa54-01ac-5ab7-a760-0517255eb533","task_revision_id":"1","upstream_id":"","short_description":"After two men robbed a local bank, they left the bank and went in different…","config":"all","split":"test","body":"{\"choices\":\"['inadmissible, because it violates the physician- patient privilege.', 'admissible, because the technician overheard the conversation.', \\\"admissible, because of the nature of the man's statements.\\\", 'admissible, because the statement was not confidential, under the circumstances.']\",\"question\":\"After two men robbed a local bank, they left the bank and went in different directions with a plan to meet three days later at the train station. Two days after the robbeiy, one of the men was involved in an auto accident. The man was rushed to the hospital and taken to the x-ray room. A doctor positioned the man so that the x-ray technician would be able to take x-rays of his lower back area. Based on the man's complaints about the area of pain, the doctor stated, \\\"It seems as if you might have a fractured pelvis. If you do, you'll have to stay in bed for several weeks. \\\" The man responded that he couldn't do that because he had to meet the other man tomorrow. A technician who was in the room at the time overheard the man's response to the doctor. Subsequently, both men are charged with robbery and conspiracy. At the man's trial, as evidence of a conspiracy, the prosecution attempts to introduce the doctor's testimony that the man said, \\\"I have to meet the other man tomorrow. \\\" The defense objects. This evidence is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}