{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72763e78-5a61-5573-a370-0b90157e3a16","task_key":"default--test--2220","task_revision_id":"1","upstream_id":"2220","short_description":"One cause of the deinstitutionalization of many psychiatric patients in the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['a decrease in the number of psychiatric hospitals.', 'an increase in the number of psychiatric patients.', 'an increase in the number of trained mental health professionals.', 'the development of new psychotherapy techniques.', 'a decrease in government funding.', 'a decrease in the stigma of mental illness.', 'the creation of new medications.', 'an increase in government funding.', 'an improvement in psychotherapy.', 'the movement for the rights of the mentally ill.']\",\"question\":\"One cause of the deinstitutionalization of many psychiatric patients in the mid-1900s was\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}