{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3ce49d4-6642-5a9e-9802-7e71e053804d","task_key":"all--test--b3ce49d4-6642-5a9e-9802-7e71e053804d","task_revision_id":"1","upstream_id":"","short_description":"One day, while visiting a plastic surgery clinic for a routine Botox treatment,…","config":"all","split":"test","body":"{\"choices\":\"[\\\"sustain the objection, because the patient's medical records are the best evidence of the doctor's negligence.\\\", 'sustain the objection as an offer to pay the medical bills.', \\\"overrule the objection, because the evidence is relevant as to the question of the doctor's negligent hiring.\\\", 'overrule the objection, because an offer to pay medical bills is an inferential admission.']\",\"question\":\"One day, while visiting a plastic surgery clinic for a routine Botox treatment, a patient received an excessive dose of Botox from a new medical assistant on the staff. Her face frozen in a grotesque grimace, she demanded to speak to the doctor. The doctor said, \\\"Listen, why don't you go over to the emergency room and send the bills to me? I'll take care of your medical expenses. \\\" The patient subsequently sued the doctor for negligent hiring of the medical assistant and for damages. At trial, she proposes to call the doctor's administrative assistant, who was present during her conversation with the doctor. The patient wants the administrative assistant to testify to the doctor's offer to pay her medical bills. The doctor's attorney objects. The trial judge should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}