{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4683e1e0-74a5-5825-991d-8bed6f3e9e4d","task_key":"all--test--4683e1e0-74a5-5825-991d-8bed6f3e9e4d","task_revision_id":"1","upstream_id":"","short_description":"When both short-run aggregate supply and aggregate demand increase which of the…","config":"all","split":"test","body":"{\"choices\":\"['The price level rises but real GDP falls.', 'Both the price level and real GDP rise.', 'The price level rises but the change in real GDP is uncertain.', 'Real GDP rises but the change in the price level is uncertain.']\",\"question\":\"When both short-run aggregate supply and aggregate demand increase which of the following can be said for certain?\",\"subject\":\"high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}