{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9dfce278-baf6-5fb3-9efc-7d468f2a406f","task_key":"all--test--9dfce278-baf6-5fb3-9efc-7d468f2a406f","task_revision_id":"1","upstream_id":"","short_description":"Your new ‘client is a 37-year-old woman whose symptoms meet the diagnostic…","config":"all","split":"test","body":"{\"choices\":\"['inform the client of the limits of confidentiality and report the unethical behavior.', 'inform the client of the limits of confidentiality and contact the other psychologist.', \\\"discuss the client's options with her and maintain her confidentiality unless she signs a release.\\\", 'contact the Ethics Committee to determine what the appropriate steps would be in this situation.']\",\"question\":\"Your new ‘client is a 37-year-old woman whose symptoms meet the diagnostic criteria for Major Depressive Disorder. After your third session with her, she tells you that she was in treatment with another psychologist but terminated after the psychologist made sexual advances toward her. She does not want you to discuss this matter with anyone. To be consistent with the provisions of the ethics codes of the American and Canadian Psychology Association, your best course of action would be to:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}