{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"825ae680-6429-5eeb-8aec-ebec6251e0cd","task_key":"default--test--3258","task_revision_id":"1","upstream_id":"3258","short_description":"Define the termsAnthropoidea,Hominidaeand Hominoidea?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Anthropoidea are the primitive primates, Hominidae are the great apes, and Hominoidea are fossil men and modern man', 'Anthropoidea are the cats, Hominidae are the dogs, and Hominoidea are the birds', 'Anthropoidea are the Old World monkeys, Hominidae are the New World monkeys, and Hominoidea are the great apes', 'Anthropoidea are the insectivores, Hominidae are the carnivores, and Hominoidea are the cetaceans', 'Anthropoidea are the higher primates, Hominidae are fossil men and modern man, and Hominoidea are great apes and man', 'Anthropoidea are the reptiles, Hominidae are the amphibians, and Hominoidea are the mammals excluding primates', 'Anthropoidea are the New World monkeys, Hominidae are the Old World monkeys, and Hominoidea include all catarrhine species', 'Anthropoidea are the great apes, Hominidae are the lesser apes, and Hominoidea are the primates excluding humans', 'Anthropoidea include all primates, Hominidae are a subgroup of rodents, and Hominoidea are the amphibians', 'Anthropoidea are the prosimians, Hominidae are the marsupials, and Hominoidea are the rodents']\",\"question\":\"Define the termsAnthropoidea,Hominidaeand Hominoidea?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}