{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45172a0c-40f0-5ff2-91cb-65f978621cc7","task_key":"all--test--45172a0c-40f0-5ff2-91cb-65f978621cc7","task_revision_id":"1","upstream_id":"","short_description":"More than a dozen species of Darwin's finches with various specialized…","config":"all","split":"test","body":"{\"choices\":\"['The various foods available on the mainland are different than those on the Galápagos Islands.', 'South American predators limited evolution of the mainland finch.', 'Reproductive isolation is not possible on the mainland.', \\\"The available niches that the Darwin's finches exploited on the Galápagos Islands were already occupied by other species of birds on the mainland.\\\"]\",\"question\":\"More than a dozen species of Darwin's finches with various specialized adaptations inhabit the Galápagos Islands. Each species possesses a specialized adaptation for obtaining food. Why is it that similar adaptations do not exist among the finches of the South American mainland, the presumed origin of the Galápagos finch ancestor?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}