{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81b9b9d3-1dc2-5157-91c0-97a1c7531545","task_key":"all--test--81b9b9d3-1dc2-5157-91c0-97a1c7531545","task_revision_id":"1","upstream_id":"","short_description":"A general contractor about to bid on a construction job for an office building…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The carpenter detrimentally relied on the general contractor's conditional promise in preparing his bid.\\\", \\\"The carpenter gave consideration for the general contractor's conditional promise to award the carpentry subcontract to the carpenter.\\\", 'The general contractor has an implied duty to deal fairly and in good faith with all bidders whose bids the general contractor used in calculating its main bid to the building owner.', \\\"The general contractor has an obligation to the owner of the building to subcontract with the carpenter because the carpenter's bid was used in calculating the general contractor's bid, and the carpenter is an intended beneficiary of that obligation.\\\"]\",\"question\":\"A general contractor about to bid on a construction job for an office building invited a carpenter and several others to bid on the carpentry work. The carpenter agreed to bid if the general contractor would agree to give the carpenter the job provided that his bid was lowest and the general contractor was awarded the main contract. The general contractor so agreed. The carpenter, incurring time and expense in preparing his bid, submitted the lowest carpentry bid. The general contractor used the carpenter's bid in calculating its own bid, which was successful. Which of the following best supports the carpenter's position that the general contractor is obligated to position that the general contractor is obligated to award the carpentry subcontract to the carpenter?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}