{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0eafa4f1-22a9-5bff-ad5d-b834ec653f56","task_key":"default--test--5474","task_revision_id":"1","upstream_id":"5474","short_description":"Accrual accounting involves accruals and deferrals. Which of the following best…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Accruals are concerned with current cash receipts and payments while deferrals are concerned with expected future cash receipts and payments.', 'Accruals are concerned with expected future cash receipts and payments while deferrals are concerned with current cash receipts and payments.', 'Both accruals and deferrals are concerned with past cash receipts and payments.', 'Both accruals and deferrals are concerned with expected future cash receipts and payments.', 'Both accruals and deferrals are concerned with past and expected future cash receipts and payments.', 'Accruals are concerned with current and future cash receipts and payments while deferrals are concerned with past and current cash receipts and payments.', 'Accruals are concerned with past cash receipts and payments while deferrals are concerned with expected future cash receipts and payments.', 'Accruals are concerned with expected future cash receipts and payments while deferrals are concerned with past cash receipts and payments.', 'Both accruals and deferrals are concerned with current cash receipts and payments.', 'Accruals are concerned with past cash receipts and payments while deferrals are concerned with current cash receipts and payments.']\",\"question\":\"Accrual accounting involves accruals and deferrals. Which of the following best describes accruals and deferrals?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}