{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d81466f8-8751-5f35-8a49-16fcded03cf5","task_key":"all--test--d81466f8-8751-5f35-8a49-16fcded03cf5","task_revision_id":"1","upstream_id":"","short_description":"A landlord orally leased a commercial building to a tenant for a 10-year term,…","config":"all","split":"test","body":"{\"choices\":\"['No, because he was a holdover tenant.', 'No, because the lease was oral.', 'Yes, because they were accessions.', 'Yes, because they were trade fixtures.']\",\"question\":\"A landlord orally leased a commercial building to a tenant for a 10-year term, which ended one year ago. At the beginning of the lease term, the tenant installed numerous appliances in the building as well as a large air-conditioning system. The tenant alone paid for these items; they were installed with the landlord's permission and were used for the tenant's commercial enterprise. At the end of the lease term, the tenant failed to vacate the premises in a timely manner and continued to pay rent while remaining in possession for another four months. Before vacating the building at the end of the fourth month, the tenant removed the appliances and the air-conditioning system that he had installed, causing no damage to the building. Was the removal by the tenant proper?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}