{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cbf0df62-1eed-5db3-ab0a-6c4ae11e84e3","task_key":"default--test--5707","task_revision_id":"1","upstream_id":"5707","short_description":"Which of the following types of audit evidence provides the least assurance of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Emails from the client discussing financial matters.', 'Prenumbered receiving reports completed by the client’s employees.', \\\"Client's self-prepared inventory count records.\\\", \\\"Verbal confirmations from the client's suppliers.\\\", \\\"Customer invoices prepared by the client's employees.\\\", 'Municipal property tax bills prepared in the client’s name.', 'Prior months’ bank statements obtained from the client.', 'Receivable confirmations received from the client’s customers.', \\\"Client's financial forecasts for the next fiscal year.\\\", 'Unofficial financial statements provided by the client.']\",\"question\":\"Which of the following types of audit evidence provides the least assurance of reliability?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}