{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48c29d7b-963a-542c-babb-7f33587ca89b","task_key":"default--test--2449","task_revision_id":"1","upstream_id":"2449","short_description":"Dr. Ryan is a psychotherapist in a small town. She receives a phone call from a…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['see the man but disclose their past relationship to a supervisor or colleague for transparency.', \\\"deny the man's request without providing a reason.\\\", 'see the man only if their relationship ended more than two years ago and she determines that it will not interfere with her objectivity.', 'see the man only if she feels she can maintain professional boundaries despite their past.', 'refer the man to a colleague.', 'continue therapy with the man but avoid discussing their past relationship.', 'see the man only if she determines that their past relationship will not interfere with her objectivity.', 'provide therapy for the man over the phone or through online sessions to maintain physical boundaries.', 'see the man only if his wife is aware of their past relationship and consents to the therapy.', 'see the man but discuss the potential for conflicts with him before beginning treatment.']\",\"question\":\"Dr. Ryan is a psychotherapist in a small town. She receives a phone call from a man she was sexually involved with several years ago. The relationship lasted only three months and ended amicably. The man is now married and is having problems unrelated to their past relationship. He has called to see if he can begin seeing Dr. Ryan in therapy. Dr. Ryan should:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}