{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1cd3caa3-17b6-5ef6-826d-87a15bab6af7","task_key":"default--test--2642","task_revision_id":"1","upstream_id":"2642","short_description":"One of the key areas of debate in the early stages of Developmental Psychology…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Secular trends.', 'Epigenesis.', 'Genotype-environment interaction.', 'Canalization?', 'Critical period.', 'Phenotypic plasticity.', 'Genomic imprinting.', 'Genetic drift.', 'Sensitive period.', 'Genetic correlation.']\",\"question\":\"One of the key areas of debate in the early stages of Developmental Psychology was the so called, Nature vs. Nurture debate. Academics agreed that there was interaction between these two factors but there was wide disagreement concerning the degree of interaction. Which of the following terms, that relate to either genetic or environmentally caused changes, describes the differences in timing of physical changes that are found in children belonging to different cohorts?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}