{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc55d29f-8b3f-54e4-8ddb-1d8b194e679c","task_key":"default--test--5915","task_revision_id":"1","upstream_id":"5915","short_description":"Which of the following factors most likely would cause a CPA not to accept a new…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"The CPA's preference for working with larger companies.\\\", \\\"The CPA's lack of understanding of the entity's operations and industry.\\\", \\\"The entity's financial year does not align with the CPA's schedule.\\\", \\\"The CPA's current workload is too heavy to accommodate a new client.\\\", \\\"Management's unwillingness to make all financial records available to the CPA.\\\", \\\"The entity is not willing to cover the CPA's travel expenses for audit purposes.\\\", \\\"The entity's location is in a different city than the CPA's office.\\\", \\\"The CPA's lack of interest in the particular industry of the entity.\\\"]\",\"question\":\"Which of the following factors most likely would cause a CPA not to accept a new audit engagement?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}