{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6a6555f-390c-578c-b94b-16f6143b8b19","task_key":"all--test--d6a6555f-390c-578c-b94b-16f6143b8b19","task_revision_id":"1","upstream_id":"","short_description":"A patient who had suffered a severe fracture of her leg was treated by an…","config":"all","split":"test","body":"{\"choices\":\"['No, because the patient has introduced evidence that the first orthopedist failed to give the care that the second orthopedist would have provided.', 'No, because the second orthopedist practices in the same town and field of specialty as the first orthopedist.', \\\"Yes, because the patient has failed to introduce evidence that the first orthopedist's care fell below the professional standard of care.\\\", 'Yes, because the second orthopedist also treated the patient and is thus not sufficiently objective to offer expert testimony.']\",\"question\":\"A patient who had suffered a severe fracture of her leg was treated by an orthopedist, who set the patient's leg and put it in a cast. When the leg continued to bother the patient six months later, she consulted a second orthopedist in the same town. The second orthopedist surgically inserted a pin to facilitate healing. The patient brought a malpractice action against the first orthopedist, claiming that he should have surgically inserted a pin at the time of initial treatment. The only evidence that the patient offered in support of her malpractice claim was the testimony of the second orthopedist, as follows: In response to the question \\\"Would you have inserted a pin initially?\\\" the second orthopedist testified, \\\"I personally would not have been satisfied that the leg would heal properly without a pin.\\\" At the close of the patient's evidence, the first orthopedist moved for judgment as a matter of law. Should the motion be granted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}