{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6653733d-8b77-574b-af75-51deee174df5","task_key":"all--test--6653733d-8b77-574b-af75-51deee174df5","task_revision_id":"1","upstream_id":"","short_description":"An entity engaged an accountant to review its financial statements in accordance…","config":"all","split":"test","body":"{\"choices\":\"['Withdrawn from the engagement because the entity has not been following GAAP.', 'Advised management to make the adjustments necessary for the account balances to conform with GAAP.', 'Modified the review report to reflect the fact that the financial statements were presented on another comprehensive basis of accounting.', 'Requested that management justify the use of the other comprehensive basis of accounting in the management representation letter.']\",\"question\":\"An entity engaged an accountant to review its financial statements in accordance with Statements on Standards for Accounting and Review Services. The accountant determined that the entity maintained its accounts on a comprehensive basis of accounting other than generally accepted accounting principles (GAAP). In this situation, the accountant most likely would have taken which of the following actions?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}