{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"455a7ed0-6ddf-50b9-9881-94a6a1256a76","task_key":"default--test--2709","task_revision_id":"1","upstream_id":"2709","short_description":"If, during a postexamination discussion with parents, a psychologist establishes…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['tell the parents that he/she cannot interfere in this matter', 'cell the parents char be/she is legally obligated to inform the pediatrician', 'tell the parents that he/she will inform the pediatrician', 'tell the parents that he/she will discuss with their previous pediatrician', 'cell the parents that it is their responsibility to inform the pediatrician', \\\"tell the parents to discuss it with the child's school counselor\\\", 'suggest the parents to inform the pediatrician themselves', \\\"inform the pediatrician without the parents' consent\\\", 'inform the pediatrician and then notify the parents of his/her actions', 'urge the parents to grant him/her permission to inform the pediatrician']\",\"question\":\"If, during a postexamination discussion with parents, a psychologist establishes that a child’s new pediatrician is apparently unaware of the child's history of brain damage. which is very important in understanding the problem situation, the psychologist should\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}