{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd52cb8a-0e29-58cb-b1a7-ffe7eb8ce30a","task_key":"all--test--dd52cb8a-0e29-58cb-b1a7-ffe7eb8ce30a","task_revision_id":"1","upstream_id":"","short_description":"The Farstate Department of Wealth Equality is a state agency dedicated to taking…","config":"all","split":"test","body":"{\"choices\":\"['The action is arbitrary and capricious, so it is unconstitutional under the strict scrutiny standard.', 'The action is not deemed done by a state actor because wealth redistribution is not an essential government function.', 'The right to public employment is a protected property interest and the employees would likely prevail because of lack of procedural due process.', 'Wealth is a quasi-suspect class, so the rule must promulgate an important governmental purpose. Eliminating poverty is important, so the rule meets this level of scrutiny.']\",\"question\":\"The Farstate Department of Wealth Equality is a state agency dedicated to taking from the rich to give to the poor. The head of the department has promulgated the following hiring rule: \\\"Because the rich are the source of all problems of society and cannot understand the plight of the poor, anyone who has a net worth over $10,000.00 is ineligible for employment with this department.\\\" Immediately, 25 employees in the department are fired due to their net worth above the designated level. They sue the state. Which of the following is the correct analysis and conclusion?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}