{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1735d3bc-b3dd-5e60-9b42-e7dabeabee35","task_key":"all--test--1735d3bc-b3dd-5e60-9b42-e7dabeabee35","task_revision_id":"1","upstream_id":"","short_description":"Subsequent to issuing a report on audited financial statements, a CPA discovers…","config":"all","split":"test","body":"{\"choices\":\"['Bring the matter to the attention of the board of directors or audit committee.', 'Withdraw the auditor’s report from those persons currently relying on it.', 'Perform alternative procedures to verify account balances.', 'Discuss the potential financial statement adjustments with client management.']\",\"question\":\"Subsequent to issuing a report on audited financial statements, a CPA discovers that the accounts receivable confirmation process omitted a number of accounts that are material, in the aggregate. Which of the following actions should the CPA take immediately?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}