{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62f8b5da-f6bb-5852-a4b8-d28dc05ea1c9","task_key":"default--test--5262","task_revision_id":"1","upstream_id":"5262","short_description":"If the objective of an auditor's test of details is to detect the overstatement…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"Customer's purchase orders to the sales journal.\\\", 'Shipping documents to the cash receipts journal.', 'Cash receipts journal to the shipping documents.', 'Sales journal to the cash receipts journal.', 'Sales journal to the shipping documents.', 'Shipping documents to the sales journal.', \\\"Customer's purchase orders to the cash receipts journal.\\\", \\\"Cash receipts journal to the customer's purchase orders.\\\", \\\"Shipping documents to the customer's purchase orders.\\\", \\\"Customer's purchase orders to the shipping documents.\\\"]\",\"question\":\"If the objective of an auditor's test of details is to detect the overstatement of sales the auditor should trace transactions from the\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}