{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bb7173b4-f148-550d-a711-66ee0c910be3","task_key":"all--test--bb7173b4-f148-550d-a711-66ee0c910be3","task_revision_id":"1","upstream_id":"","short_description":"A 22-year-old woman contacts a medical student and asks if he would like to join…","config":"all","split":"test","body":"{\"choices\":\"['He can date her because he was a medical student, not a physician, when he contributed to her care', 'He can date her because she is no longer his patient', 'He can date her, but only after at least 1 year has passed since he treated her', 'He cannot date her because she was once his psychiatric patient']\",\"question\":\"A 22-year-old woman contacts a medical student and asks if he would like to join her for dinner. The student met the woman when he was assigned to her care during her 2-week hospitalization for treatment of major depressive disorder. He has not treated or seen the patient since she was discharged from the hospital. He is attracted to this former patient and would be interested in dating her. Which of the following is the most appropriate action by the medical student regarding this patient's invitation?\",\"subject\":\"professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}