{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1efb131-4663-545e-8627-84a47734705a","task_key":"default--test--5475","task_revision_id":"1","upstream_id":"5475","short_description":"Prior to commencing the compilation of financial statements of a nonissuer, an…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"Consult with the entity's legal team about any pending litigation or legal matters.\\\", \\\"Conduct a thorough internal audit of the entity's financial operations.\\\", 'Verify that the financial information supplied by the entity agrees with the books of original entry.', 'Perform preliminary analytical procedures to identify accounts that may represent specific risks relevant to the engagement.', \\\"Make inquiries of management concerning the entity's procedures used in adjusting and closing the books of account.\\\", \\\"Verify the entity's tax records and filings.\\\", \\\"Perform a risk assessment of the entity's major financial transactions in the last fiscal year.\\\", \\\"Obtain an understanding of any specialized financial reporting frameworks and practices used in the entity's industry.\\\", \\\"Assess the entity's compliance with relevant financial regulations and standards.\\\", \\\"Review the entity's insurance policies and coverage.\\\"]\",\"question\":\"Prior to commencing the compilation of financial statements of a nonissuer, an accountant is required to\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}