{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"07c98584-a7a8-5950-a91c-104af6082083","task_key":"default--test--6299","task_revision_id":"1","upstream_id":"6299","short_description":"A \"dished face\" profile is often associated with","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['a small chin due to micrognathia.', 'a recessive maxilla due to failure of elongation of the cranial base.', 'a sunken cheek due to maxillary hypoplasia.', 'an enlarged frontal bone due to hydrocephaly.', 'a protruding mandible due to reactivation of the condylar cartilage by acromegaly.', 'defective development of the maxillary air sinus.', 'a wide-set eye due to hypertelorism.', 'a protruding forehead due to craniosynostosis.', 'a flattened forehead due to plagiocephaly.', 'an enlarged mandible due to gigantism.']\",\"question\":\"A \\\"dished face\\\" profile is often associated with\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}