{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1debd3c-7f85-57ff-93b9-1533a512da5f","task_key":"default--test--5183","task_revision_id":"1","upstream_id":"5183","short_description":"When there has been a change in accounting principles, but the effect of the…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Issue a qualified opinion due to the change.', 'Refer to the note in the financial statements that discusses the change.', 'Explicitly state whether the change conforms with GAAP.', 'Include a paragraph detailing the change in the notes to the financial statements.', 'Issue an adverse opinion due to the change.', \\\"Request that the client restate the prior year's financial statements.\\\", 'Discontinue the audit due to the change in accounting principles.', 'Not refer to the change in the auditor’s report.', 'Refer to the change in an emphasis-of-matter paragraph.', 'Require the client to disclose the change in a separate financial statement.']\",\"question\":\"When there has been a change in accounting principles, but the effect of the change on the comparability of the financial statements is not material, the auditor should\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}