{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b3251a6-7540-5ffa-909d-eb9dfac2c835","task_key":"all--test--3b3251a6-7540-5ffa-909d-eb9dfac2c835","task_revision_id":"1","upstream_id":"","short_description":"A contractor agreed to remodel a homeowner's garage for $5,000. Just before the…","config":"all","split":"test","body":"{\"choices\":\"['No, because the court must ascertain the meaning of the agreement from the terms of the written contract.', \\\"No, because the oral agreement was merely part of the parties' negotiations.\\\", 'Yes, because the contract is ambiguous.', 'Yes, because the time limit is an additional term that does not contradict the partially integrated written contract.']\",\"question\":\"A contractor agreed to remodel a homeowner's garage for $5,000. Just before the parties signed the one-page written contract, the homeowner called to the contractor's attention the fact that the contract did not specify a time of completion. The parties orally agreed but did not specify in the contract that the contractor would complete the work in 60 days, and then they both signed the contract. The contract did not contain a merger clause. The contractor failed to finish the work in 60 days. The homeowner has sued the contractor for breach of contract. Is the court likely to admit evidence concerning the parties' oral agreement that the work would be completed in 60 days?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}