{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6924415b-20f1-598c-918f-921f916f3026","task_key":"default--test--1001","task_revision_id":"1","upstream_id":"1001","short_description":"A property owner owned a large building in the city. The owner leased the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Privity of estate and privity of contract exist between the property owner and the shopkeeper.', 'Privily of estate and privity of contract exist between the shopkeeper and the corporation.', 'Privily of estate, but not privily of contract, exists between property owner and the corporation.', 'Neither privily of estate nor privily of contract exists between the property owner and the nonprofit organization.']\",\"question\":\"A property owner owned a large building in the city. The owner leased the building to a shopkeeper for a period of 20 years at a rental of $10,000 per month. The leasehold agreement between property owner and shopkeeper provided that the latter was not permitted \\\"to assign this lease to anyone except a corporation with an €˜A' credit rating from a well- known credit rating company. \\\" One month later, the shopkeeper leased the premises to a corporation that did not have the required credit rating. The shopkeeper-corporation lease was for a period of five years, with a rental of $15,000 per month, payable by the corporation to the shopkeeper. In addition, the corporation agreed to abide \\\"by all of the terms and conditions of the lease between the property owner and the shopkeeper. \\\"One year later, the corporation leased the premises to a non-profit organization for the balance of the term of the corporation €”shopkeeper lease. The non-profit organization took possession of the said premises the same day that the corporation vacated its occupancy. Pursuant to the corporation €”nonprofit leasehold agreement, the latter was obligated to pay a monthly rental of $17,500 directly to the corporation. The non-profit organization has a €˜B' credit rating with the well-known credit rating company. Which of the following is not correct regarding the legal relationship(s) of the various parties?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}