{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"928cd86d-b2fc-5a6a-b2f8-ff09e23f716c","task_key":"default--test--2341","task_revision_id":"1","upstream_id":"2341","short_description":"The mother of a 31-year-old client that you have been seeing for six months…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['advise her in general terms about how she can help her son.', 'suggest she should engage in regular conversation with her son about his feelings.', 'suggest she should provide emotional support for her son.', 'tell her that you cannot discuss her son’s condition with her.', 'ask her to encourage her son to talk more openly about his therapy sessions.', 'tell her to read more about the condition her son is suffering from.', \\\"disclose detailed information about her son's progress in therapy.\\\", \\\"recommend she seeks her own therapy to better understand her son's situation.\\\", 'suggest that she make an appointment so you can discuss the matter in person.', 'suggest that she come to therapy with her son at his next appointment.']\",\"question\":\"The mother of a 31-year-old client that you have been seeing for six months calls to check on her son's progress. She says that she is very concerned about her son and wants to know if there is anything she can do to help him. As an ethical psychologist, you:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}