{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60a200b4-e2f1-559f-bb01-fcca63f791f1","task_key":"all--test--60a200b4-e2f1-559f-bb01-fcca63f791f1","task_revision_id":"1","upstream_id":"","short_description":"Since the 1996 World Food Summit, how has the number of food insecure people in…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The Summit's goal of cutting the number of hungry people in half by 2015 was achieved.\\\", \\\"The number decreased, but not by nearly enough to meet the Summit's goal.\\\", 'The number increased slightly.', 'Because of rising food prices, the number increased dramatically.']\",\"question\":\"Since the 1996 World Food Summit, how has the number of food insecure people in the world changed?\\n\",\"subject\":\"nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}