{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c277bdd3-a2c4-5f7c-954d-75095648fcbf","task_key":"default--test--5567","task_revision_id":"1","upstream_id":"5567","short_description":"An entity prepares its financial statements on its income tax basis. The…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Is required to be included in the balance sheet.', 'Is required to be included in the income statement.', \\\"Need not be quantified and included in either the notes to the financial statements or the auditor's report.\\\", \\\"Is required to be included in the balance sheet and the auditor's report.\\\", 'Is required to be included only in the notes to the financial statements.', \\\"Is required to be included in the income statement, the balance sheet, and the auditor's report.\\\", \\\"Is required to be included in the auditor's report and the balance sheet.\\\", \\\"Is required to be included only in the auditor's report.\\\", 'Is required to be included in the income statement and the notes to the financial statements.', \\\"Is required to be included both in the notes to the financial statements and the auditor's report.\\\"]\",\"question\":\"An entity prepares its financial statements on its income tax basis. The accompanying notes include a summary of significant accounting policies that discusses the basis of presentation and describes how that basis differs from GAAP. The dollar amount of the effects of the difference between the income tax basis and GAAP\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}