{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a5a739a-5270-5462-9ffa-3e76a71cfebe","task_key":"all--test--6a5a739a-5270-5462-9ffa-3e76a71cfebe","task_revision_id":"1","upstream_id":"","short_description":"An aluminum can manufacturer entered into negotiations with an aluminum sheeting…","config":"all","split":"test","body":"{\"choices\":\"['The agreement constituted a valid modification of their June 1 contract.', 'The agreement was unenforceable, since the supplier was under a pre-existing duty to supply the manufacturer with the sheeting under their June 1 contract.', 'The agreement constituted an enforceable reformation of their June 1 contract.', 'The agreement was unenforceable, since there was no new consideration.']\",\"question\":\"An aluminum can manufacturer entered into negotiations with an aluminum sheeting supplier to supply the manufacturer's requirements of aluminum sheeting. After extensive discussions, the parties signed the following agreement on June 1:\\\"The supplier agrees to supply all of the manufacturer's requirements of aluminum sheeting for a period of two years beginning on August 1, at a price of $3. 00 per unit. \\\"On June 16, the manufacturer notified the supplier that it would need a delivery of 2,000 units by August 1 to accommodate its needs. The supplier informed the manufacturer that it did not contemplate such high requirements, since its plant's capacity was only 2,800 per month. Moreover, the supplier pointed out that in order to meet the manufacturer's order of 2,000 units, it would probably lose two long-time customers, whose outstanding orders would have to be canceled. After a week of negotiations, on June 23 the supplier orally agreed to deliver 2,000 units of sheeting to the manufacturer by August 1. The parties then signed the following contract:\\\"The supplier agrees to supply all of the manufacturer's requirements of aluminum sheeting for a period of two years beginning August 1, at a price of $3. 50 per unit. The manufacturer agrees that it will require a minimum of 800 units in every month of the agreement. \\\"On June 25 the supplier notified its two longtime customers that it was canceling their August orders (of 800 units) because of other contract commitments. Which of the following is the most accurate statement regarding the written agreement between the parties on June 23?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}