{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b752e47f-ecd1-573c-94c0-550b2ca628d8","task_key":"all--test--b752e47f-ecd1-573c-94c0-550b2ca628d8","task_revision_id":"1","upstream_id":"","short_description":"An embryologist studying the development of a vertebrate organism from the…","config":"all","split":"test","body":"{\"choices\":\"['Ontogeny recapitulates phylogeny.', 'Early embryos display identical features of their class, order, and species.', 'An early human embryo has features in common with early fish and avian embryos.', 'A human embryo displays features of adult fish and birds in the course of its development.']\",\"question\":\"An embryologist studying the development of a vertebrate organism from the zygote stage to fetus would be justified in drawing which of the following conclusions?\",\"subject\":\"college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}