{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7093301-7018-55c7-820c-f5673340f4b0","task_key":"all--test--c7093301-7018-55c7-820c-f5673340f4b0","task_revision_id":"1","upstream_id":"","short_description":"An accountant has been engaged to compile a nonissuer's financial statements…","config":"all","split":"test","body":"{\"choices\":\"['Disclaim an opinion on the financial statements and advise the board of directors that the financial statements should not be relied upon.', \\\"Inform management that the engagement can proceed only if distribution of the accountant's compilation report is restricted to internal use.\\\", 'Determine the effects of the deficiencies and add a separate paragraph to the compilation report that describes the deficiencies and their effects.', 'Withdraw from the compilation engagement and provide no further services concerning these financial statements.']\",\"question\":\"An accountant has been engaged to compile a nonissuer's financial statements that contain several misapplications of accounting principles and unreasonable accounting estimates. Management is unwilling to revise the financial statements and the accountant believes that modification of the standard compilation report is not adequate to communicate the deficiencies. Under these circumstances the accountant should\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}