{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bd1fd3c7-c474-5ae5-b1f6-db08bfeebfaa","task_key":"all--test--bd1fd3c7-c474-5ae5-b1f6-db08bfeebfaa","task_revision_id":"1","upstream_id":"","short_description":"Which of the following policies is not true of economic policy?","config":"all","split":"test","body":"{\"choices\":\"['Economic policies must take into account the incentives of not only the leaders, but the populations intended to influence.', 'Policymakers can incentivize security-promoting behaviours in others through the use of sanctions and aid.', 'Trade policies can entice leaders to change behaviours in exchange for access to markets or trade preferences, and advance security.', 'Concern with poverty is a humanitarian issue that has no direct link to economic science, although it might be alleviated through aid.']\",\"question\":\"Which of the following policies is not true of economic policy?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}