{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d22f0a90-b0f1-560f-8ac6-03d5dd604d4c","task_key":"all--test--d22f0a90-b0f1-560f-8ac6-03d5dd604d4c","task_revision_id":"1","upstream_id":"","short_description":"Which of the following best explains why babies have poor vision for the first…","config":"all","split":"test","body":"{\"choices\":\"['The nodes of Ranvier have not yet formed.', 'The neural connections to the primary visual cortex are not fully connected.', 'The axons are covered in tight coats of lipids, which impede neural firing.', 'The synaptic cleft of the neuron is filled with an aqueous humor.']\",\"question\":\"Which of the following best explains why babies have poor vision for the first few weeks of life?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}