{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ddae08ed-5236-5c77-88d2-3c13062f4727","task_key":"default--test--5991","task_revision_id":"1","upstream_id":"5991","short_description":"An auditor is using statistical sampling in testing whether cash disbursements…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Overall risk assessment of the audit.', 'Planned reliance on the prescribed control.', 'Tolerable rate of deviations.', 'Assessed level of control risk.', 'Use of statistical sampling in the audit.', 'Assessed level of detection risk.', 'Level of inherent risk.', 'Sample size for further testing.', 'Period of time for which the audit is conducted.', 'Planned reliance on substantive tests.']\",\"question\":\"An auditor is using statistical sampling in testing whether cash disbursements were properly authorized. The sample results indicate that the sample rate of deviation plus the allowance for sampling risk exceeds the tolerable rate. Under these circumstances the auditor most likely would reduce the\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}