{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f9284de-7ef6-5ba1-b957-f68aa03d7df8","task_key":"all--test--6f9284de-7ef6-5ba1-b957-f68aa03d7df8","task_revision_id":"1","upstream_id":"","short_description":"While performing an audit of the financial statements of a company for the year…","config":"all","split":"test","body":"{\"choices\":\"['There is a broad indication of misappropriation of assets.', 'There is an indication of theft of the entity’s assets.', 'There is an indication of embezzling receipts.', 'There is a broad indication of financial reporting fraud.']\",\"question\":\"While performing an audit of the financial statements of a company for the year ended December 31, year 1, the auditor notes that the company’s sales increased substantially in December, year 1, with a corresponding decrease in January, year 2. In assessing the risk of fraudulent financial reporting or misappropriation of assets, what should be the auditor’s initial indication about the potential for fraud in sales revenue?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}