{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"16a29284-4357-5b72-8652-412e3137e4ea","task_key":"all--test--16a29284-4357-5b72-8652-412e3137e4ea","task_revision_id":"1","upstream_id":"","short_description":"The secondarily altricial condition of modern human babies may have been an…","config":"all","split":"test","body":"{\"choices\":\"['the brain size of hominids had not grown for more than a million years, making it difficult for babies to walk.', 'it allowed for bipedalism and independence from the mother at an earlier age.', 'it allowed for subsequent growth of the brain, since bipedalism had resulted in a narrowed birth canal in the female pelvis.', 'it allowed for growth of the brain to nearly adult size prior to birth, since bipedalism had resulted in a wider birth canal in the female pelvis.']\",\"question\":\"The secondarily altricial condition of modern human babies may have been an evolutionary solution, in that:\",\"subject\":\"prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}