{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fdcf90b-3c08-53dd-b701-dcd2d9cd0f04","task_key":"all--test--1fdcf90b-3c08-53dd-b701-dcd2d9cd0f04","task_revision_id":"1","upstream_id":"","short_description":"The economy is in a deep recession. Given this economic situation which of the…","config":"all","split":"test","body":"{\"choices\":\"['Expansionary policy would only worsen the recession.', 'Expansionary policy greatly increases aggregate demand if investment is sensitive to changes in the interest rate.', 'Contractionary policy is the appropriate stimulus for investment and consumption.', 'If the demand for money is perfectly elastic expansionary monetary policy might be quite effective.']\",\"question\":\"The economy is in a deep recession. Given this economic situation which of the following statements about monetary policy is accurate?\",\"subject\":\"high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}