{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a28c8f60-dd9e-5c6f-ab04-fa012b514641","task_key":"default--test--2621","task_revision_id":"1","upstream_id":"2621","short_description":"Current legal and ethical standards require that a treatment record be kept for…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"Diagnoses; initial findings; dates of service and billing; patient's fantasies and cognitions\\\", 'Diagnoses; treatment plan; dates of service and billing; progress in the treatment plan', \\\"Initial findings; treatment plan; dates of service and billings; patient's progress\\\", \\\"initial findings: consent of patient's cognitions, reveries, and fantasies; dares of service and billings patient's progress\\\", \\\"initial findings; dates of service and billing; patient's progress\\\", 'diagnoses; initial findings; treatment plan; dates of service and billing; progress in the treatment plan', \\\"Diagnoses; initial findings; treatment plan; consent of patient's cognitions, reveries, and fantasies; dates of service and billings\\\", \\\"diagnoses; initial findings; content of patient's cognitions, reveries, and fantasies; dates of service and billing; progress in the treatment plan\\\", \\\"Diagnoses; initial findings; content of patient's cognitions, reveries, and fantasies; progress in the treatment plan\\\", \\\"Initial findings; consent of patient's cognitions, reveries, and fantasies; dates of service and billings; patient's progress\\\"]\",\"question\":\"Current legal and ethical standards require that a treatment record be kept for each patient. At the minimum, this treatment record should include\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}