{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d3c1358-da1a-5e44-bdf5-c191cc658078","task_key":"default--test--5478","task_revision_id":"1","upstream_id":"5478","short_description":"Which of the following applies to an accountant conducting a review of interim…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The accountant is required to rectify any discrepancies found during the review.', 'The accountant must conduct an in-depth investigation of all financial records.', 'The accountant must maintain independence in mental attitude in all matters relating to the engagement.', 'The accountant must express an opinion on the financial statements taken as a whole.', 'The accountant is required to take on the role of financial advisor in addition to conducting the review.', 'The accountant must perform an audit of all financial transactions.', 'The accountant must report directly to the board of directors.', 'The accountant is required to provide advice on financial decisions based on the review.', 'The accountant must obtain sufficient appropriate evidence by performing procedures to afford a reasonable basis for an opinion.']\",\"question\":\"Which of the following applies to an accountant conducting a review of interim financial information?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}