{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f8e335ab-c171-5f21-8d4b-b91511a61504","task_key":"default--test--5256","task_revision_id":"1","upstream_id":"5256","short_description":"Which of the following procedures would an auditor generally perform regarding…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Perform a complete re-audit of the financial statements.', 'Compare the latest available interim financial statements with the statements being audited.', \\\"Verify the client's tax returns for the previous year.\\\", 'Test internal control activities that were previously reported to management as inadequate.', \\\"Review the client's budget plans for the next fiscal year.\\\", \\\"Analyze the client's sales records for the first quarter of the new year.\\\", \\\"Examine the client's payroll records for several months after the year end.\\\"]\",\"question\":\"Which of the following procedures would an auditor generally perform regarding subsequent events?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}