{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d86f6d34-c8b9-5bb3-b722-75ba49e6807f","task_key":"default--test--5412","task_revision_id":"1","upstream_id":"5412","short_description":"An auditor of a nonissuer should design tests of details to ensure that…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The planned level of assurance at the relevant assertion level.', 'The planned level of detection risk.', \\\"The effectiveness of the organization's risk management.\\\", 'Management’s assertions that internal controls exist and are operating efficiently.', 'Management’s assertions about the future profitability of the company.', 'The financial statements are free from material misstatement.', 'The planned level of control risk.', \\\"The organization's compliance with laws and regulations.\\\", \\\"The accuracy of the organization's financial reporting.\\\", 'The effectiveness of internal controls.']\",\"question\":\"An auditor of a nonissuer should design tests of details to ensure that sufficient audit evidence supports which of the following?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}