{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f533a134-9533-57b6-9595-c10506d8c09f","task_key":"default--test--2095","task_revision_id":"1","upstream_id":"2095","short_description":"A female psychologist provides a 2-month course of brief behavior therapy for a…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Ethically, as long as she does not provide therapy to the client again in the future', 'Ethically, because she waited an appropriate amount of time after termination before starting a relationship', 'Ethically because it had been 6 months since termination', 'Ethically, because the client initiated the relationship', 'Ethically according to professional standards, but the psychologist exhibited poor professional judgment', 'Unethically, because she engaged in a sexual relationship with a former client, which is prohibited under any circumstances', 'Probably ethically since there was no coercion involved and the relationship was consensual', 'Unethically, because she should have referred the client to another psychologist before starting a relationship']\",\"question\":\"A female psychologist provides a 2-month course of brief behavior therapy for a driving phobia to a male client. Six months after termination they meet at an art opening reception and begin to date. Over the next few months, the relationship progresses and they become sexual. In this situation, the psychologist has acted:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}