{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db62cc96-8dd2-5c95-8b7c-e70768a81e2f","task_key":"default--test--2137","task_revision_id":"1","upstream_id":"2137","short_description":"After three sessions with a therapy client, Dr. Leonard Lykowski realizes that…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['consult with another psychologist to determine whether or not to continue seeing the client in therapy.', 'Seek personal therapy to manage his feelings and continue working with the client.', 'refer the client to another therapist after discussing the reason with her.', 'recognize that his feelings are the result of countertransference and continue to work with the client.', 'Express his hostility toward the client during the session.', 'Terminate the therapy sessions without any explanation.', 'Discuss his personal situation with the client to create transparency.', 'Ignore his feelings and continue working with the client.', 'refer the client to another therapist without telling the client the reason for the referral.', 'Ask the client to find another therapist without providing a reason.']\",\"question\":\"After three sessions with a therapy client, Dr. Leonard Lykowski realizes that he’s feeling somewhat hostile toward the client because she reminds him of his wife who he’s currently divorcing. Dr. Lykowski’s best course of action would be to:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}