{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"355a73b7-e5be-54ba-aa1f-d03b4be7b8ae","task_key":"default--test--1444","task_revision_id":"1","upstream_id":"1444","short_description":"A nephew inherited a large parcel of unimproved land from his uncle. In need of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Nothing, because the nephew did not breach the brokerage agreement.', \\\"$25,000, or the commission equivalent of 5 percent on the sale of the property for $500,000, because all conditions precedent to the nephew's duty to pay the commission were substantially fulfilled when the nephew and the developer entered into the land sale contract.\\\", \\\"$10,000, as a partial commission due to the agent's work done before the sale was cancelled.\\\", '$12,500, or half the commission, because the sale reached the halfway point when the nephew and the developer signed the contract.', 'Nothing, because as a third-party beneficiary of the contract between the nephew and the developer, the agent can enforce the contract only against the developer, but not against the nephew.', \\\"$25,000, or the commission equivalent of 5 percent on the sale of the property for$500,000, because fulfillment of the consummation condition was prevented by an event beyond the agent's control.\\\", '$25,000, because the agent provided a buyer and thus fulfilled his part of the contract.', 'Nothing, because the sale did not go through and hence no commission is due.', \\\"Nothing, because the consummation condition has not been fulfilled, and the nephew's refusal to sue the developer will not excuse that condition.\\\", '$25,000, because the agent upheld his end of the contract in making his best efforts to find a buyer for the parcel.']\",\"question\":\"A nephew inherited a large parcel of unimproved land from his uncle. In need of cash, the nephew decided to sell the parcel. He contacted a real estate agent in the area about listing the parcel for sale. The nephew and the agent entered into a valid written contract whereby the agent promised to undertake best efforts to find a buyer for the parcel. The contract also provided that the agent was to be paid a commission of 5 percent on the gross sale price following the consummation of the sale and transfer of title. The agent succeeded in finding a buyer for the parcel. The agent notified the nephew that he had found a developer who wanted to purchase the parcel for $500,000. The agent handed the nephew a real estate sales contract, signed by the developer, in which the developer agreed to pay $500,000 for the purchase of the parcel. The nephew then signed the agreement himself. However, before consummation of the sale and transfer of title, the developer, without cause, repudiated the contract. Despite the agent's insistence, the nephew refused to either sue the developer to enforce the land sale contract or pay the agent his commission. If the agent sues the nephew for breach of the brokerage agreement, which of the following, if anything, is the agent's proper measure of recovery?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}