{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"961cdd91-118b-5e8c-a0b0-772b6cc4854e","task_key":"default--test--5841","task_revision_id":"1","upstream_id":"5841","short_description":"In performing substantive tests regarding the granting of stock options to…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Interview senior management to determine their knowledge of stock options.', \\\"Confirm with the issuer's legal counsel whether the options were legally granted.\\\", 'Review the public records of the SEC to determine whether the options were properly reported.', \\\"Trace the authorization for the options granted to the board of directors' approval.\\\", 'Confirm with those members of management as to whether they are actually option holders.', \\\"Review the issuer's tax records to determine the tax implications of the stock options.\\\", \\\"Verify the existence of option holders in the issuer's payroll and human resources records.\\\", \\\"Review the minutes of the board of directors' meetings to determine if the options were discussed.\\\", \\\"Analyze the financial reports to determine the impact of the stock options on the company's profitability.\\\", \\\"Inspect the company's employee handbook to verify the policy on stock options.\\\"]\",\"question\":\"In performing substantive tests regarding the granting of stock options to senior management of an issuer an auditor most likely would\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}