{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bb34944-f5d9-55ad-aad9-e9780132850e","task_key":"all--test--9bb34944-f5d9-55ad-aad9-e9780132850e","task_revision_id":"1","upstream_id":"","short_description":"DDT was once considered a \"silver bullet\" that would permanently eradicate…","config":"all","split":"test","body":"{\"choices\":\"['Larger doses of DDT should have been applied.', 'All habitats should have received applications of DDT at about the same time.', 'The frequency of DDT application should have been higher.', 'None of the individual insects should have possessed genomes that made them resistant to DDT.']\",\"question\":\"DDT was once considered a \\\"silver bullet\\\" that would permanently eradicate insect pests. Today, instead, DDT is largely useless against many insects. Which of these would have been required for this pest eradication effort to be successful in the long run?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}