{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66f13e22-1292-5b4e-a746-02e11abb7d6b","task_key":"default--test--5842","task_revision_id":"1","upstream_id":"5842","short_description":"To which of the following parties will a CPA be liable if the CPA fraudulently…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['No No', 'To both parties, but only if the CPA had knowledge of the fraudulent activity', 'Only to corporate bondholders if they can prove reliance on the misstated financial statements', 'Yes Yes', 'No Yes', 'To neither party, unless they can prove reliance and financial loss due to the misstated financial statements', 'Yes No', 'Only to corporate shareholders if they can prove reliance on the misstated financial statements', 'To both parties, but only if the CPA had a direct contractual relationship with them', 'To both parties, but only if they can prove financial loss due to the misstated financial statements']\",\"question\":\"To which of the following parties will a CPA be liable if the CPA fraudulently issues an unqualified opinion on a corporation's materially misstated financial statements? Corporate shareholders, Corporate bondholders\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}