{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62e93fc7-f4e0-59b2-aa99-e546e5a78997","task_key":"default--test--2002","task_revision_id":"1","upstream_id":"2002","short_description":"Dr. Delbert Dinwoody is currently treating a 35-year-old client who has received…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['acceptable as long as the client and wife are not separated or in the process of divorce.', 'acceptable as long as the wife is not receiving mental health services from another professional.', '\\\"unacceptable because in this situation, the wife is “vulnerable to undue influence.\\\"\\\"\\\"', 'acceptable only if the wife has also been diagnosed with Alcohol Dependence.', \\\"acceptable as long as he believes couple therapy will benefit the client's progress in dealing with his alcohol problem.\\\", 'unacceptable because the wife should seek her own professional help.', '\\\"unacceptable because doing so creates a \\\"\\\"conflict of interest.”\\\"', 'acceptable only if the wife gives her written consent to participate in the therapy.', \\\"unacceptable because it may create a bias in the therapist's approach towards the client.\\\", \\\"unacceptable because it infringes on the client's individual therapy process.\\\"]\",\"question\":\"Dr. Delbert Dinwoody is currently treating a 35-year-old client who has received a diagnosis of Alcohol Dependence. With the client's permission, Dr. Dinwoody invites the client's wife to participate in couple therapy. Inviting the wife to participate in therapy is:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}