{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f92cac5f-d4d7-591b-959f-4ee674bf73bd","task_key":"default--test--6145","task_revision_id":"1","upstream_id":"6145","short_description":"In the discharge checklist recommended by the Department of Health (2004), who…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The ward clerk, GP, and patient.', 'The nurse, ward clerk, and consultant.', 'The GP, patient, and ward clerk.', 'The GP, consultant, and named nurse.', 'The consultant, patient, and named nurse.', 'The consultant, named nurse, and GP.', 'The nurse, patient, and ward clerk.', 'The nurse, GP, and ward clerk.', 'The nurse, consultant, and GP.', 'The ward clerk, consultant, and named nurse.']\",\"question\":\"In the discharge checklist recommended by the Department of Health (2004), who should be involved in completing the form?\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}