{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0011a0eb-a93c-5fde-a93d-923666cace1e","task_key":"default--test--1910","task_revision_id":"1","upstream_id":"1910","short_description":"A landlord, an owner in fee simple, leased a three- story house to a tenant for…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the student, because the new owner should have notified him about the taxes.', 'the new owner.', 'the landlord, because he never objected to the assignment.', 'the student, because he was not aware of the original lease agreement.', 'the new owner, because the student was not part of the original lease agreement.', 'the student, because the covenant to pay taxes will not \\\"run with the land,\\\" unless the intent is clearly expressed in the original lease.', 'the tenant, because he was the one who originally agreed to pay the taxes.', 'the student, because he did not contract with the tenant to pay the taxes on the property.', 'the student, because the covenant to pay taxes is merely collateral and does not \\\"run with the land. \\\"']\",\"question\":\"A landlord, an owner in fee simple, leased a three- story house to a tenant for a term of 10 years. By the terms of the lease, the tenant expressly covenanted to pay a monthly rental of $300 and to pay the taxes on the premises during the term of the lease. The lease also stipulated that the tenant, as lessee, may not assign or sublease the said premises. The tenant and his family lived in the house for two years. Then the tenant, still owing the landlord three months back rent, assigned his leasehold interest in the property to a student, who was unaware of the prohibition against assignments. This written assignment expressly provided that the student would pay the landlord the monthly rental of $300, but was silent concerning the payment of taxes. The landlord never objected to this assignment. Six months after the assignment, the landlord conveyed his property interest in the premises to a new owner. The student failed to pay the taxes on the property. The new owner, after paying the taxes, brought suit against the student for the amount paid. Judgment should be for\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}