{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"698dda7c-b08d-5064-80db-520ec1b5d1a1","task_key":"all--test--698dda7c-b08d-5064-80db-520ec1b5d1a1","task_revision_id":"1","upstream_id":"","short_description":"A landlord, the owner of a two-story dwelling house, leased it completely…","config":"all","split":"test","body":"{\"choices\":\"[\\\"succeed, because the tenant is estopped to deny the landlord's title.\\\", 'succeed, because of his security interest in the property.', 'fail, because the purchase by the tenant vitiated any further contractual obligations.', 'fail, because the landlord was under a duty to keep the demised premises in reasonably good repair.']\",\"question\":\"A landlord, the owner of a two-story dwelling house, leased it completely furnished to a tenant for a 1 0-year period. Toward the end of the seventh year of the term, a violent storm blew off several of the roof shingles. The tenant noticed a leak through the roof where the shingles had blown off, but didn't make any repairs. A month later, a severe rain storm occurred, and water leaked through the roof, causing damage to the valuable parquet floors in the two rooms below. Before the term of his lease ended, the tenant discovered that the landlord had not paid his taxes on the property, so he purchased the property through a sheriff's sale. The tenant refuses to make any further rental payments on the property. If the landlord brings suit against the tenant for payment of rent in arrears, he would most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}