{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05e3a5db-2dec-5386-be69-e26692aa67d5","task_key":"all--test--05e3a5db-2dec-5386-be69-e26692aa67d5","task_revision_id":"1","upstream_id":"","short_description":"Which statement about the beak size of finches on the island of Daphne Major…","config":"all","split":"test","body":"{\"choices\":\"['Each bird evolved a deeper, stronger beak as the drought persisted.', 'Each bird developed a deeper, stronger beak as the drought persisted.', \\\"Each bird's survival was strongly influenced by the depth and strength of its beak as the drought persisted.\\\", 'Each bird that survived the drought produced only offspring with deeper, stronger beaks than seen in the previous generation.']\",\"question\":\"Which statement about the beak size of finches on the island of Daphne Major during prolonged drought is true?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}