{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86bd96b2-d552-5fbd-831d-11a856acf56b","task_key":"all--test--86bd96b2-d552-5fbd-831d-11a856acf56b","task_revision_id":"1","upstream_id":"","short_description":"While performing interim audit procedures of accounts receivable, numerous…","config":"all","split":"test","body":"{\"choices\":\"['Move detailed analytical procedures from year-end to interim.', 'Increase the dollar threshold of vouching customer invoices', 'Send negative accounts receivable confirmations instead of positive accounts receivable confirmations', 'Use more experienced audit team members to perform year-end testing']\",\"question\":\"While performing interim audit procedures of accounts receivable, numerous unexpected errors are found resulting in a change of risk assessment. Which of the following audit responses would be most appropriate?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}