{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"edff01a5-aba4-589b-aa87-ef1b4ee2de25","task_key":"all--test--edff01a5-aba4-589b-aa87-ef1b4ee2de25","task_revision_id":"1","upstream_id":"","short_description":"After three sessions with a therapy client, Dr. Leonard Lykowski realizes that…","config":"all","split":"test","body":"{\"choices\":\"['refer the client to another therapist without telling the client the reason for the referral.', 'refer the client to another therapist after discussing the reason with her.', 'recognize that his feelings are the result of countertransference and continue to work with the client.', 'consult with another psychologist to determine whether or not to continue seeing the client in therapy.']\",\"question\":\"After three sessions with a therapy client, Dr. Leonard Lykowski realizes that he’s feeling somewhat hostile toward the client because she reminds him of his wife who he’s currently divorcing. Dr. Lykowski’s best course of action would be to:\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}