{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"892d3fb3-2984-5293-a716-0f327880d364","task_key":"default--test--2298","task_revision_id":"1","upstream_id":"2298","short_description":"To treat Zoe's anorexia nervosa, her doctors put her on intravenous feeding…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Behaviorist.', 'cognitive behavioral.', 'Humanistic.', 'Psychoanalytic.', 'Socio-cultural.', 'psychodynamic.', 'eclectic.', 'biological.', 'Gestalt.', 'Existential.']\",\"question\":\"To treat Zoe's anorexia nervosa, her doctors put her on intravenous feeding tubes, tried to change her irrational belief that she was too fat, and discussed how her early family relationships may have contributed to her current problems. This approach would best be classified as\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}