{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb11181f-be7f-5d60-bcd2-9847d9ae3374","task_key":"all--test--cb11181f-be7f-5d60-bcd2-9847d9ae3374","task_revision_id":"1","upstream_id":"","short_description":"A company offered to sell several loads of landscaping stones at a specified…","config":"all","split":"test","body":"{\"choices\":\"['The offeror will prevail because the posting of the acceptance finalized a contract between the parties.', 'The offeree will prevail because it never accepted the offer on the same terms specified.', 'The offeror will prevail because it shipped the goods when it received the first acceptance.', \\\"The offeree will prevail because it rejected the offer when it called the offeror's manager.\\\"]\",\"question\":\"A company offered to sell several loads of landscaping stones at a specified price and itemized terms. The offeree sent a notification to the offeror stating, \\\"We have received your offer to sell landscaping stone per specified terms this date.\\\" The offeror shipped the goods upon receiving that notification. On the date of shipment, the offeree posted an acceptance form with slight modifications to the terms. After the goods arrived, the offeree called the offeror's manager and told him that the company was unable to accept the offer at this time, he was shipping the goods back, and to \\\"ignore any paperwork we sent.\\\" The offeror sued the offeree for breach of contract. Who will prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}