{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a346d48-90e1-5b9d-82ec-04c4ebd9257f","task_key":"default--test--5341","task_revision_id":"1","upstream_id":"5341","short_description":"Which of the following dental topics is developmentally appropriate for…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Growth of permanent teeth', 'Correct toothbrushing technique', 'The science behind root canal treatment', 'Understanding the need for dentures', 'Teeth whitening procedures', 'Correct use of dental floss', 'Introduction to dental implants', 'The process of tooth extraction', 'Correct use of incisors while eating', 'Evolution of the human tooth']\",\"question\":\"Which of the following dental topics is developmentally appropriate for toddlers?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}