{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3943c37-40a8-5315-bba0-595011a187eb","task_key":"default--test--5557","task_revision_id":"1","upstream_id":"5557","short_description":"Before accepting an audit engagement a CPA should evaluate whether conditions…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"There are significant differences between the entity's forecasted financial statements and the financial statements to be audited.\\\", \\\"There is a high turnover rate among the entity's senior management.\\\", 'The CPA discovers that the entity has been consistently late in paying its bills.', 'The CPA will not be permitted to have access to sensitive information regarding the salaries of senior management.', \\\"The entity's management has been involved in recent legal disputes.\\\", 'The CPA becomes aware of the existence of related party transactions while reading the draft financial statements.', 'There have been substantial inventory write-offs just before the year end in each of the past four years.', \\\"The entity's financial statements show a consistent pattern of losses.\\\"]\",\"question\":\"Before accepting an audit engagement a CPA should evaluate whether conditions exist that raise questions as to the integrity of management. Which of the following conditions most likely would raise such questions?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}