{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b26ba1d2-3996-5a8e-ac70-6d388c50fb9a","task_key":"all--test--b26ba1d2-3996-5a8e-ac70-6d388c50fb9a","task_revision_id":"1","upstream_id":"","short_description":"Was T.R. Malthus correct in concluding that population growth will outstrip food…","config":"all","split":"test","body":"{\"choices\":\"['No, technological and institutional innovations have permitted food supplies to more than keep pace with population growth.', 'Yes, it is only because of widespread famines that population has not overwhelmed food supplies altogether.', 'No, but it is only because of the introduction of genetically modified food that supplies have kept pace with population growth.', 'No, but supplies are not likely to keep pace with population growth over the next 10 years.']\",\"question\":\"Was T.R. Malthus correct in concluding that population growth will outstrip food production growth?\\n\",\"subject\":\"nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}