{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f00a2929-9b84-5134-88c1-dc21eeb86cd5","task_key":"default--test--929","task_revision_id":"1","upstream_id":"929","short_description":"A buyer expressed interest in purchasing an industrial air-conditioning system…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the oral price term is relevant to whether the writing should be reformed.', 'No, because the oral price term would contradict an express term in the written agreement.', 'Yes, because the buyer can refuse to pay more than the orally agreed price.', 'Yes, because the oral agreement is considered more binding than the written agreement.', \\\"Yes, because the seller cannot increase the price without the buyer's consent.\\\", 'Yes, because the written agreement did not contain a merger clause.', 'No, because the court does not consider oral agreements.', 'No, because the buyer assumed the risk of any mistake as to price.', 'No, because the written agreement takes precedence over the oral agreement.', 'No, because the seller has the right to increase the price in the written agreement.']\",\"question\":\"A buyer expressed interest in purchasing an industrial air-conditioning system manufactured by the seller. The parties agreed orally on a price of $100,000 for the system, but continued to negotiate over several points. When all matters regarding the air-conditioning system were finally settled, the parties signed a written agreement. It provided that the price for the system, which would be delivered on June 1, would be $110,000. The written agreement, a lengthy form contract, did not contain a merger clause. The seller delivered the system on June 1, but the buyer refused to pay more than $100,000, citing the earlier oral agreement as to price. The seller sued the buyer for the additional $10,000 under the written agreement. Is the court likely to admit the evidence of the orally agreed price of $100,000?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}