{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2934bac5-26b3-5d87-a9fe-441ee7f96715","task_key":"all--test--2934bac5-26b3-5d87-a9fe-441ee7f96715","task_revision_id":"1","upstream_id":"","short_description":"A freight company offered to purchase a large order of tires from a wholesale…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because a revocation of an offer requires explicit words of rejection in order to be enforced.', 'Yes, because the seller must be able to rely on a definite order of goods.', 'No, because the words of revocation were clear enough to communicate that the deal was off.', 'No, because a buyer is never in breach of an agreement when it rejects goods prior to their shipment.']\",\"question\":\"A freight company offered to purchase a large order of tires from a wholesale tire distributor for specified terms. Before receiving an answer, the purchasing agent called the wholesaler to state, \\\"We are confronting a critical financial crisis and cannot afford to make the purchase.\\\" The seller sent an acceptance form the next day, and shipped the goods with an invoice. When the purchasing company did not pay and shipped the goods back, the seller sued for his lost profit. The seller claimed that the purchasing agent's words did not clearly communicate a revocation of the offer. Will the seller prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}