{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c848f0ea-717d-5f7f-819e-31d9cc8c82f6","task_key":"all--test--c848f0ea-717d-5f7f-819e-31d9cc8c82f6","task_revision_id":"1","upstream_id":"","short_description":"The President announced that a newly created military rapid deployment force…","config":"all","split":"test","body":"{\"choices\":\"['constitutional, because the President, in this instance, has not been called by Congress into actual service as Commander-in-Chief.', 'constitutional, because of Congressional power under the War Powers Act.', \\\"unconstitutional, because of the President's authority to execute the laws of the United States.\\\", \\\"unconstitutional, because of the President's authority as Commander-in-Chief of the Armed Forces.\\\"]\",\"question\":\"The President announced that a newly created military rapid deployment force would be engaging in joint training exercises with the military forces in a foreign country. Following the President's announcement, Congress enacted a statute that unequivocally prohibited \\\"the U. S. Armed Forces from conducting military exercises with foreign military forces unless Congress has received notice of such intention to take such action at least three months before the joint military exercises become effective. \\\"This statute is most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}