{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ee71a96-e546-5120-aeba-cb94ffd02536","task_key":"all--test--4ee71a96-e546-5120-aeba-cb94ffd02536","task_revision_id":"1","upstream_id":"","short_description":"A buyer, located on the west coast contacted a seller, located on the east…","config":"all","split":"test","body":"{\"choices\":\"['succeed, because the risk of loss was on the seller.', \\\"succeed, because the carrier was the seller's agent.\\\", 'not succeed, because the risk of loss was on the buyer.', 'not succeed, because of frustration of purpose.']\",\"question\":\"A buyer, located on the west coast contacted a seller, located on the east coast, about purchasing flanges that the buyer used in their manufacturing process. The parties entered into a contract whereby the seller agreed to deliver to the buyer 1,000 flanges for $10,000. The contract stipulated that it was FOB at the seller's place of business. The seller then delivered the flanges to a delivery company, which was to transport them to the buyer. While en route, the vehicle that was transporting the flanges was involved in an accident and the flanges were destroyed. When the buyer received notice of the accident, the buyer immediately contacted the seller and demanded that it ship replacement flanges. The seller refused to do so. In an action by the buyer against the seller for breach of contract, the buyer will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}