{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5d62f90-381e-56ea-a704-3ea277d0755b","task_key":"all--test--a5d62f90-381e-56ea-a704-3ea277d0755b","task_revision_id":"1","upstream_id":"","short_description":"A buyer expressed interest in purchasing an industrial air-conditioning system…","config":"all","split":"test","body":"{\"choices\":\"['No, because the buyer assumed the risk of any mistake as to price.', 'No, because the oral price term would contradict an express term in the written agreement.', 'Yes, because the oral price term is relevant to whether the writing should be reformed.', 'Yes, because the written agreement did not contain a merger clause.']\",\"question\":\"A buyer expressed interest in purchasing an industrial air-conditioning system manufactured by the seller. The parties agreed orally on a price of $100,000 for the system, but continued to negotiate over several points. When all matters regarding the air-conditioning system were finally settled, the parties signed a written agreement. It provided that the price for the system, which would be delivered on June 1, would be $110,000. The written agreement, a lengthy form contract, did not contain a merger clause. The seller delivered the system on June 1, but the buyer refused to pay more than $100,000, citing the earlier oral agreement as to price. The seller sued the buyer for the additional $10,000 under the written agreement. Is the court likely to admit the evidence of the orally agreed price of $100,000?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}