{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5456ae1f-6bfb-5911-aefe-5a8b178e431b","task_key":"default--test--5123","task_revision_id":"1","upstream_id":"5123","short_description":"Which of the following factors most likely would cause an accountant not to…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"Management's decision to not include the financial statements in a written personal financial plan.\\\", 'Indications that reports of asset misappropriation are not investigated by management.', \\\"The accountant's lack of experience in the entity's industry.\\\", \\\"A lack of segregation of duties in the entity's accounting and payroll departments.\\\", \\\"The entity's intention to include more disclosures than required by GAAP.\\\", \\\"Management's acknowledgement that the financial statements will be included in a written personal financial plan.\\\", \\\"The entity's lack of an internal audit department.\\\"]\",\"question\":\"Which of the following factors most likely would cause an accountant not to accept an engagement to compile the financial statements of a nonissuer?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}