{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d824c4c-0fbc-57cf-ba50-224d9b319874","task_key":"default--test--6470","task_revision_id":"1","upstream_id":"6470","short_description":"Which of the following is NOT one of the most rewarding aspects of being a…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Sharing wisdom and life experiences', 'Having a second chance at parenting', 'Watching grandchildren grow and develop', 'Having others to wait on you', 'Reliving their own childhood through their grandchildren', 'Having an excuse to indulge in childish activities', 'Spoiling the grandchildren', 'Feelings of immortality', 'Being able to spend quality time with family', 'Remembering when they were grandchildren']\",\"question\":\"Which of the following is NOT one of the most rewarding aspects of being a grandparent as reported by grandparents?\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}