{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f05ab23-7ee3-59bb-8433-8c34a9bcd1bc","task_key":"default--test--5570","task_revision_id":"1","upstream_id":"5570","short_description":"Joe is the trustee of a trust set up for his father. Under the Internal Revenue…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['May sign the return but must include a separate statement explaining his relationship to the trust.', 'May not sign the return unless he is also named as a co-trustee.', 'Can only sign the return if he is also a certified public accountant.', 'Is considered a tax return preparer only if he is also the beneficiary of the trust.', 'May not sign the return unless he receives additional compensation for the tax return.', 'Must obtain the written permission of the beneficiary prior to signing as a tax return preparer.', 'Is considered a tax return preparer because his father is the grantor of the trust.', 'Is not considered a tax return preparer.', \\\"Is not considered a tax return preparer unless he personally benefits from the trust's assets.\\\", 'Is considered a tax return preparer and must pay a fee to the IRS for this service.']\",\"question\":\"Joe is the trustee of a trust set up for his father. Under the Internal Revenue Code, when Joe prepares the annual trust tax return, Form 1041, he\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}