{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8883ef2c-9c6f-5b79-91af-9eb6c75db039","task_key":"all--test--8883ef2c-9c6f-5b79-91af-9eb6c75db039","task_revision_id":"1","upstream_id":"","short_description":"A landlord is the owner in fee simple of a tract of land on which is situated a…","config":"all","split":"test","body":"{\"choices\":\"[\\\"the landlord, because the tenant's contractual obligation under the lease survived the assignments.\\\", 'the landlord, because he did not object to the assignments.', 'the tenant, because the doctor, as assignee, would only be held liable.', 'the tenant, because his assignment to the attorney constituted a novation, thereby extinguishing his obligation to pay rent.']\",\"question\":\"A landlord is the owner in fee simple of a tract of land on which is situated a large office building. The landlord leases the land and building thereon to a tenant for a term of seven years, commencing on August 1, 2001, and terminating at midnight on July 31, 2008. The lease contains the following provisions:\\\"(1) The tenant covenants to pay the rent of $750 per month on the first day of each month. \\\"After three years of the lease had expired, the tenant assigned the entire balance of the lease period to an attorney, who took immediate possession of the leased property. Then in 2005, the attorney assigned his leasehold interest to a doctor. The doctor went into possession, but failed to pay any rent for two months. After the doctor failed to make his rental payments for the first two months of his lease, the landlord brought suit against the tenant to recover for the unpaid rent. Judgment should be for\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}