{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff04e215-a371-5fd3-bcd6-3318f6048f58","task_key":"default--test--1736","task_revision_id":"1","upstream_id":"1736","short_description":"A landlord was the owner of a vacant warehouse. On May 1, the landlord leased…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"landowner's duty of care to third parties cannot be shifted to a tenant by the tenns of a lease.\\\", 'tenant is responsible for all injuries occurring on the property once possession has been transferred.', 'tenant cannot waive the implied warranty of commercial habitability.', 'landlord is absolved of all responsibilities once a lease agreement is signed.', \\\"landlord's failure to repair the defect is not relevant to the case.\\\", 'covenant to indemnify by a tenant in favor of a land owner is against public policy.', \\\"tenant's employee was not protected under the leasehold agreement.\\\", 'landowner is strictly liable for injuries occurring on his property.', \\\"landlord's duty of care to a tenant can be transferred through a lease agreement.\\\", \\\"tenant's acknowledgment of the state of disrepair absolves the landlord of any liability.\\\"]\",\"question\":\"A landlord was the owner of a vacant warehouse. On May 1, the landlord leased the warehouse to the tenant for a term of five years with a renewable option. The leasehold agreement provided in part:\\\"The tenant hereby acknowledges that certain areas of the leased premises are in a state of disrepair and unsafe for the conduct of business. Nonetheless, the tenant agrees to assume full responsibility for the necessary repairs. Furthermore, the tenant agrees to indemnify the landlord for any loss resulting from the condition of the said premises. \\\"Under the terms of the lease, the landlord delivered possession of the warehouse to the tenant on May 2. On that date, an employee of the tenant was working in the warehouse. He was moving some equipment into the warehouse when a section of the ceiling collapsed and fell on him. At the time the landlord and the tenant entered into their lease, only the landlord was aware that the ceiling was defective and needed to be repaired. Although the landlord could have remedied the defect before delivering possession to the tenant, the landlord didn't perform any repair work on the ceiling. The employee initiated suit against the landlord to recover damages resulting from his injury. There are no applicable statutes. If the employee prevails it will be because a\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}