{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e2d11d5-8b8d-539f-bb23-76b4d2ee3f4a","task_key":"default--test--3226","task_revision_id":"1","upstream_id":"3226","short_description":"What social and behavioral characteristics are paralleled in the primates, in…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Aquatic lifestyle, migration, carnivorous diet, and aggressive behavior.', 'Monogamous pair bonding, bioluminescence for communication, amphibious lifestyle, and metamorphosis.', 'Predatory behavior, nocturnal lifestyle, hibernation, and solitary living.', 'Burrowing lifestyle, herbivorous diet, mating rituals, and solitary living.', 'Use of tools, construction of complex structures, agricultural practices, and development of written language.', 'Arboreal living, fruit-based diet, nocturnal foraging, and use of echolocation.', 'Solitary living, desert habitat adaptation, venomous defense mechanisms, and egg laying.', 'Seasonal migration, hibernation, herbivorous diet, and complex burrowing systems.', 'Grouping lifestyles, similar life cycles, birth spacing, dependency on the mother, play groups, division of labor between sexes, omnivorous diet, hunting and food sharing, communication, and occupying a specific geographic area.', 'Pack hunting, diurnal lifestyle, camouflage techniques, and cold climate adaptation.']\",\"question\":\"What social and behavioral characteristics are paralleled in the primates, in particular chimpanzees, gorillas and man?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}