{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc616c64-9bf1-5155-a1b3-57ad2be06bf3","task_key":"default--test--1282","task_revision_id":"1","upstream_id":"1282","short_description":"A housing corporation owned a tract of land and prepared a development plan to…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The community association has no authority to enforce the Declaration of Covenants.', 'The Declaration of Covenants does not specifically mention the landscaper as a liable party.', 'There is no privity of estate between the gardener and the landscaper.', 'The landscaper is not a direct party to the contract and therefore should not be liable for the payment.', 'The annual assessment is not enforceable as it was not included in the deed of sale.', \\\"Because the charge constitutes a lien, there is no personal obligation on the landscaper's part.\\\", 'There is not privity of contract between the housing corporation and the community association.', 'The landscaper was not aware of the annual assessment at the time of purchase.', 'There is no privity of contract between the gardener and the landscaper.']\",\"question\":\"A housing corporation owned a tract of land and prepared a development plan to divide the land into 100 lots and create a residential community on the tract of land. The Declaration of Covenants created the community association, an administrative entity that would administer and enforce the regulations and restrictions recited among the covenants. One of the regulations set forth in the Declaration of Covenants reads:\\\"Each purchaser, by the acceptance of a deed therefore, promises to pay the community association an annual assessment or charge to be determined on the basis of the valuation of each individual lot and the improvements thereon. Nonpayment of any annual assessment or charge when due shall result in a lien upon the parcel of the property. \\\"A gardener, the owner of lot 29 in the development, sold his land to a landscaper with a deed containing no restrictions. The community association, pursuant the Declaration of Covenants, sues the landscaper to collect the annual assessment for lot 29. Which of the following is the best argument for the landscaper?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}