{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84c7109f-6520-560d-b2c5-d68ecad30f8e","task_key":"default--test--2933","task_revision_id":"1","upstream_id":"2933","short_description":"An embryologist studying the development of a vertebrate organism from the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The development of a vertebrate organism from the zygote stage to fetus does not follow a consistent pattern.', 'The development of a vertebrate organism from the zygote stage to fetus displays a random pattern of evolutionary history.', 'The zygote stage of a vertebrate organism is identical to the fetus stage.', 'Ontogeny recapitulates phylogeny.', 'Early embryos display identical features of their class and order, but not their species.', 'An early human embryo has features in common with early fish and avian embryos.', 'Early embryos display identical features of their class, order, and species.', 'A human embryo displays features of adult fish and birds in the course of its development.', 'An early human embryo has features that are identical to adult fish and avian features.', 'A vertebrate organism develops identical features of its class, order, and species only during the fetus stage.']\",\"question\":\"An embryologist studying the development of a vertebrate organism from the zygote stage to fetus would be justified in drawing which of the following conclusions?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}