{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c60662c1-25dd-59bb-b3ec-5bd0bd4abfbf","task_key":"default--test--5912","task_revision_id":"1","upstream_id":"5912","short_description":"Which of the following statements should be included in a practitioner’s report…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['A statement that the practitioner performed an audit of internal control over financial reporting.', 'A statement that the practitioner has provided absolute assurance on the financial statements.', 'A statement of scope limitation that will qualify the practitioner’s opinion.', 'A statement of negative assurance based on procedures performed.', \\\"A statement describing the practitioner's personal opinion on the financial statements.\\\", 'A statement that the practitioner has performed a comprehensive financial analysis.', 'A statement that the practitioner performed an examination of prospective financial statements.', 'A statement referring to standards established by the International Accounting Standards Board.', 'A statement referring to standards established by the AICPA.', 'A statement claiming full responsibility for the accuracy of the financial statements.']\",\"question\":\"Which of the following statements should be included in a practitioner’s report on the application of agreed-upon procedures?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}