{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"34038012-a74b-5ec6-94be-e0c4546fcb6e","task_key":"default--test--2142","task_revision_id":"1","upstream_id":"2142","short_description":"Your new ‘client is a 37-year-old woman whose symptoms meet the diagnostic…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Refuse to continue therapy until she allows you to report the unethical behavior.', 'Inform the client of the limits of confidentiality and insist on reporting the unethical behavior.', \\\"discuss the client's options with her and maintain her confidentiality unless she signs a release.\\\", 'contact the Ethics Committee to determine what the appropriate steps would be in this situation.', \\\"Contact the police to report the other psychologist's behavior.\\\", 'Discuss the situation with your supervisor before proceeding.', 'inform the client of the limits of confidentiality and contact the other psychologist.', 'inform the client of the limits of confidentiality and report the unethical behavior.', 'Inform the client that you cannot continue therapy due to a conflict of interest.', 'Continue therapy without addressing the issue.']\",\"question\":\"Your new ‘client is a 37-year-old woman whose symptoms meet the diagnostic criteria for Major Depressive Disorder. After your third session with her, she tells you that she was in treatment with another psychologist but terminated after the psychologist made sexual advances toward her. She does not want you to discuss this matter with anyone. To be consistent with the provisions of the ethics codes of the American and Canadian Psychology Association, your best course of action would be to:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}