{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c512a51e-2498-56d0-bc2b-9c6ea9f6611c","task_key":"all--test--c512a51e-2498-56d0-bc2b-9c6ea9f6611c","task_revision_id":"1","upstream_id":"","short_description":"A landlord was the owner of a large, high-rise apartment building in a…","config":"all","split":"test","body":"{\"choices\":\"['there was nothing in the lease regarding liability for fire.', 'the two tenants did not own an interest in the property.', \\\"the jurisdiction has rejected the common law view on the tenant's duty to pay rent.\\\", 'the landlord did not contract to convey the property to the two tenants.']\",\"question\":\"A landlord was the owner of a large, high-rise apartment building in a Midwestern city. On June 1, 2007, two tenants took possession of a three- bedroom apartment in the landlord's building under a three-year lease at a rental of $1,200 per month. Their lease (as all other leases given by the landlord) contained the following provisions:\\\"The term of this lease shall be three years from the date hereof as long as all the agreements herein shall be faithfully performed. \\\"The two tenants lived in the apartment for two years. On June 10, 2009, however, a fire destroyed the apartment building. As a result, all the apartments in the building were rendered uninhabitable. After the two tenants were dispossessed from their apartment, the landlord brought suit against them to recover the rent due for the balance of the lease. The two tenants claim that they are no longer liable for rent or any other obligations under the lease. The landlord €” tenants leasehold contract contained no provision regarding liability for fire. If the decision is in favor of the two tenants, it will most likely be because\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}