{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25392e83-9deb-55d7-b266-fedcee6f88b7","task_key":"default--test--5190","task_revision_id":"1","upstream_id":"5190","short_description":"An auditor of a nonissuer may not issue a qualified opinion when","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"The auditor does not have sufficient understanding of the entity's industry.\\\", \\\"The auditor is unable to verify the entity's reported revenue.\\\", 'The auditor lacks independence with respect to the entity.', \\\"Management prevents the auditor from observing the entity's inventory.\\\", 'The entity omits the statement of cash flows from its financial statements.', 'The entity has a significant amount of debt.', 'The entity refuses to provide the auditor with access to its financial records.', \\\"The entity's financial statements are not prepared in accordance with generally accepted accounting principles.\\\", \\\"The auditor has a personal relationship with a member of the entity's board.\\\", \\\"The auditor's report refers to the work of an actuary.\\\"]\",\"question\":\"An auditor of a nonissuer may not issue a qualified opinion when\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}