{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c8e5f72-2460-5aa7-a1fe-fc5fa74fd1e1","task_key":"all--test--8c8e5f72-2460-5aa7-a1fe-fc5fa74fd1e1","task_revision_id":"1","upstream_id":"","short_description":"The buyer asked a manufacturer to make him 100,000 widgets for $1,000,000. The…","config":"all","split":"test","body":"{\"choices\":\"['Yes, the statute of frauds must be enforced when it involves goods over $500.', 'Yes, the manufacturer did not protect its interests by agreeing to an oral contract and it must pay for its bad business practices.', 'No, because there is an exception for goods that are specially made for a buyer.', 'No, the law will enforce the \\\"benefit of the bargain\\\" rule.']\",\"question\":\"The buyer asked a manufacturer to make him 100,000 widgets for $1,000,000. The buyer intended to retail them by mail order sales. The widgets had to have the buyer's logo imprinted on them. The manufacturer ordered the raw material and made a new widget mold that contained buyer's logo. The buyer wanted quick delivery of at least 10,000 right away. The manufacturer rushed the order and sent 10,000 conforming widgets two days later. The buyer paid for the widgets but rejected delivery on the remaining 90,000 widgets. The manufacturer filed suit and claimed damages for breach of contract. Defendant buyer filed a motion to dismiss the complaint on the basis that the statute of frauds was violated in that it was a contract for over $500 that was unenforceable under state law. Will the court likely grant the motion to dismiss?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}