{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfe1b7d2-df11-58a1-9bb2-d5728c3d72d6","task_key":"all--test--dfe1b7d2-df11-58a1-9bb2-d5728c3d72d6","task_revision_id":"1","upstream_id":"","short_description":"A pedestrian is suing a defendant for injuries she suffered to her leg when the…","config":"all","split":"test","body":"{\"choices\":\"['sustain the objection, as hearsay not within any recognized exception.', 'sustain the objection, as violative of the physician-patient privilege.', 'overrule the objection, as a statement made for the purpose of diagnosis or treatment.', 'overrule the objection, as a statement against interest.']\",\"question\":\"A pedestrian is suing a defendant for injuries she suffered to her leg when the defendant ran the pedestrian over with his motorcycle. On cross- examination of the pedestrian's physician, the defendant's attorney asked him whether or not the pedestrian had told him that the leg that she claimed was injured had been permanently stiff as a result of a former field hockey injury the pedestrian suffered in high school. Upon objection by the pedestrian's attorney, the trial judge should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}