{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6c89b2e-becf-5df0-9ead-cbe64f8fbd07","task_key":"default--test--2441","task_revision_id":"1","upstream_id":"2441","short_description":"When consulting with a colleague about a “therapeutic impasse” you are having…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['it is only necessary to advise the client of the consultation if the client specifically asks about it.', 'it is only necessary to advise the client of the consultation if the colleague is outside of your practice.', 'it is unnecessary to advise the client of the consultation as long as only information that is relevant to the impasse is discussed.', 'it is unnecessary to advise the client of the consultation if the colleague is a licensed professional.', \\\"it is unnecessary to advise the client of the consultation as long as the client's identity is not revealed.\\\", 'it is necessary to get a signed waiver of confidentiality from the client before discussing confidential information with the consultant.', \\\"it is necessary to advise the client of the consultation only if the client's identity is revealed.\\\", 'it is always necessary to get written approval from the client before discussing their case with a colleague.', 'it is always necessary to advise the client about the consultation.', 'it is unnecessary to advise the client of the consultation as long as you do not discuss the specifics of the therapy sessions.']\",\"question\":\"When consulting with a colleague about a “therapeutic impasse” you are having with a therapy client:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}