{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3095186-b66b-5571-ad0b-c5575626531c","task_key":"all--test--b3095186-b66b-5571-ad0b-c5575626531c","task_revision_id":"1","upstream_id":"","short_description":"When the Founders met in 1787 to write the Constitution, their primary objective…","config":"all","split":"test","body":"{\"choices\":\"['establish a bill of rights that would protect citizens from the government', 'create a national court system for hearing disputes between citizens of different states', 'return to the states those powers that the Articles of Confederation had given to the central government', 'replace the weak central government created by the Articles of Confederation with a stronger one']\",\"question\":\"When the Founders met in 1787 to write the Constitution, their primary objective was to\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}