{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f48d9d3b-fb1c-58d6-a7da-83a9a4b861a5","task_key":"default--test--5472","task_revision_id":"1","upstream_id":"5472","short_description":"Which of the following procedures would an accountant least likely perform…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"Assessing the company's risk management procedures.\\\", \\\"Make inquiries of management about actions taken at board of directors' meetings.\\\", 'Identify internal control activities that are likely to prevent or detect material misstatements.', 'Study the relationships of financial statement elements expected to conform to predictable patterns.', \\\"Reviewing the company's tax returns for the period\\\", 'Interviewing employees about their job responsibilities', \\\"Evaluating the company's compliance with regulations\\\", \\\"Checking for mathematical accuracy in the company's ledgers\\\", 'Compare the financial statements with anticipated results in budgets and forecasts.']\",\"question\":\"Which of the following procedures would an accountant least likely perform during an engagement to review the financial statements of a nonissuer?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}