{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"259272e0-a790-5fea-a459-2aae00eaf246","task_key":"default--test--5785","task_revision_id":"1","upstream_id":"5785","short_description":"Which of the following factors most likely would be considered an inherent…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"The effectiveness of the entity's external audit.\\\", \\\"The complexity of the entity's electronic order processing system.\\\", 'The number of employees in the entity.', \\\"The entity's market share.\\\", 'The lack of resources to monitor internal controls.', \\\"The ineffectiveness of the entity's audit committee.\\\", \\\"The physical location of the entity's main office.\\\", 'Collusion of employees in circumventing internal controls.', \\\"The entity's annual revenue.\\\", 'The lack of an annual financial audit.']\",\"question\":\"Which of the following factors most likely would be considered an inherent limitation to an entity's internal control?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}