{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b05bbbe3-5fcb-572f-b888-4f01276e4fc7","task_key":"all--test--b05bbbe3-5fcb-572f-b888-4f01276e4fc7","task_revision_id":"1","upstream_id":"","short_description":"While treating a client, you decide that you want to consult with a colleague to…","config":"all","split":"test","body":"{\"choices\":\"['you must always obtain permission from the client before talking to a consultant.', \\\"you must obtain permission from the client only if you will be revealing the client's identity to the consultant.\\\", \\\"you don't have to obtain permission from the client as long as he/she has already signed a general waiver.\\\", '\\\"you do not have to obtain permission from the client under any circumstances since the consultant is “clearly connected\\\"\\\" with the case.\\\"']\",\"question\":\"While treating a client, you decide that you want to consult with a colleague to discuss a specific aspect of the case that is giving you trouble. According to APA’s Ethics Code and the Canadian Code of Ethics:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}