{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"083ff105-32fd-5f8e-8708-1a04d42e4973","task_key":"default--test--2626","task_revision_id":"1","upstream_id":"2626","short_description":"A non-custodial parent asks Dr. Maxine Miller, a school psychologist, for the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['unethically, but within her professional rights.', 'legally, but not within her professional rights.', 'within her professional rights, but not legally.', 'ethically, but illegally.', 'illegally but ethically.', 'illegally, but within her professional rights.', 'illegally and unethically.', 'ethically, but not within her professional rights.', 'legally but unethically.', 'legally and ethically.']\",\"question\":\"A non-custodial parent asks Dr. Maxine Miller, a school psychologist, for the results of the tests she recently administered to his 5th grade son. If Dr. Miller complies with the father’s request, she will have acted:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}