{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af98a927-6c16-5292-a419-741a620f24d5","task_key":"all--test--af98a927-6c16-5292-a419-741a620f24d5","task_revision_id":"1","upstream_id":"","short_description":"Under which of the following circumstances does a president usually experience a…","config":"all","split":"test","body":"{\"choices\":\"['I only', 'I and II only', 'I and III only', 'II and III only']\",\"question\":\"Under which of the following circumstances does a president usually experience a reduction in political power? I. The president is in the final two years of his or her second term II. The number and severity of international crises is increasing III. Different parties control Congress and the White House IV. The economy is strong and the president's approval ratings are high\",\"subject\":\"high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}