{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6911fc0-1552-5e23-8310-6fd9e94886ff","task_key":"default--test--5782","task_revision_id":"1","upstream_id":"5782","short_description":"Subsequent to issuing a report on audited financial statements, a CPA discovers…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Withdraw the auditor’s report from those persons currently relying on it.', 'Contact the legal department to assess potential liabilities.', 'Request the client to reconfirm all accounts receivable.', 'Ignore the omission as it is not significant.', 'Resign from the audit engagement.', 'Amend the financial statements without informing anyone.', \\\"Issue a revised auditor's report to all relevant parties.\\\", 'Perform alternative procedures to verify account balances.']\",\"question\":\"Subsequent to issuing a report on audited financial statements, a CPA discovers that the accounts receivable confirmation process omitted a number of accounts that are material, in the aggregate. Which of the following actions should the CPA take immediately?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}