{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a06e9b30-2677-567c-a2f5-a2f30b8bc2da","task_key":"default--test--5847","task_revision_id":"1","upstream_id":"5847","short_description":"An auditor may provide an issuer client any of the following nonaudit services…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"Nonaudit services that are directly related to the operations of the issuer's business.\\\", 'Nonaudit services related to the preparation of financial statements.', 'Nonaudit services to perform financial information systems design and implementation.', 'Services that the issuer did not recognize as nonaudit services at the time of the engagement.', 'Nonaudit services that involve providing advice on legal matters.', 'Nonaudit services with revenues in aggregate of less than 5% of the total revenues paid by the issuer to the auditor during the fiscal year in which the nonaudit services are provided.', 'Nonaudit services that were promptly brought to the attention of, and approved by, the audit committee prior to the completion of the audit.', \\\"Nonaudit services related to the issuer's tax planning and compliance.\\\"]\",\"question\":\"An auditor may provide an issuer client any of the following nonaudit services without impairing independence and without obtaining the preapproval of the audit committee, except\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}