{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e94cd15-b25d-5b13-9804-379177c09682","task_key":"default--test--1731","task_revision_id":"1","upstream_id":"1731","short_description":"An investor owned a building in a city. This was the only piece of real estate…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['A joint tenancy was created.', 'The tenant became the freeholder of the property.', \\\"The tenant's equitable three-year term became a legal three-year term.\\\", 'The tenant became a subtenant of the investor.', 'A tenancy at sufferance was created.', 'A periodic tenancy from month-to-month was created.', \\\"The tenant's three-year term became a periodic tenancy.\\\", 'A tenancy at will was created.', 'A periodic tenancy from year-to-year was created.']\",\"question\":\"An investor owned a building in a city. This was the only piece of real estate that the investor owned. The three-story building had a store on the ground floor and apartments on the other two floors. The investor entered into a leasehold agreement with a tenant, who would lease the first floor, where she planned to open a sporting goods store. After identifying the parties, the operative words of the lease were as follows: \\\"Landlord hereby agrees to lease for the three years the first floor of his building in the city to tenant, reserving unto said landlord annual rental in the sum of $12,000 payable in advance in monthly installments of $1,000. \\\"After the tenant took possession and the investor accepted her rent payment for the first floor of the building, which of the following most accurately describes the legal relationship between the parties?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}