{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0d495c67-5fc4-5937-a66a-6741c70bf61c","task_key":"default--test--5633","task_revision_id":"1","upstream_id":"5633","short_description":"Which of the following procedures does a CPA usually perform when reviewing the…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Make inquiries of management concerning restrictions on the availability of cash balances.', \\\"Examine trend analysis to determine the appropriateness of the CPA's assessment of detection risk.\\\", 'Conduct a physical inventory count to verify existence and condition.', \\\"Analyze the company's market position and competitive advantage.\\\", \\\"Evaluate management's plans for dealing with negative trends and financial difficulties.\\\", 'Inspect contracts for potential legal issues or obligations.', \\\"Review the entity's compliance with tax laws.\\\", \\\"Evaluate the company's hiring practices and employee benefits packages.\\\", \\\"Assess the entity's IT systems and cyber security measures.\\\", \\\"Communicate deficiencies in the design of internal control to the entity's audit committee.\\\"]\",\"question\":\"Which of the following procedures does a CPA usually perform when reviewing the financial statements of a nonissuer?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}