{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2a090a4-d2d0-5ca3-9fe0-ac922f768a51","task_key":"all--test--f2a090a4-d2d0-5ca3-9fe0-ac922f768a51","task_revision_id":"1","upstream_id":"","short_description":"An owner of a large, three-story office building planned to renovate the…","config":"all","split":"test","body":"{\"choices\":\"['only in quantum meruit, because of the doctrifle of commercial frustration.', 'only in quantum meruit, because by demanding $80,000 the contractor, in effect, repudiated his contract with the owner.', '$75,000 only, because that was the contract price.', \\\"$80,000, because the contractor reasonably relied to his detriment on the electrician's bid in formulating his job estimate.\\\"]\",\"question\":\"An owner of a large, three-story office building planned to renovate the building and requested several contractors to submit bids to perform the work. One of the contractors in turn sought bids from several subcontractors to determine the overall cost of the job. An electrician submitted a bid to do the electrical work to the contractor, which the contractor used in computing the bid he sent to the owner. The contractor notified the electrician to that effect. Two weeks later, the contractor submitted his bid to the owner, in which he offered to perform the renovation work for $75,000. The next day, the electrician notified the contractor that he was unable to perform the electrical work on the owner's project. Two days later, the owner sent the contractor a signed confirmation letter wherein she stated, \\\"I hereby accept your offer to perform the renovation work for the quoted cost of $75,000. \\\"Thereafter, the contractor hired another electrician to complete the electrical work at a cost of $5,000 above the first electrician's bid. The contractor explained the situation to the owner and informed her that the overall cost of the job would have to be increased to $80,000. The owner responded that she would hold the contractor to his original bid of $75,000 and would not be responsible for any additional costs. The contractor then performed the renovation work, but the owner has not yet paid him anything. In an action by the contractor against the owner for the services rendered, the contractor will probably be able to recover\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}