{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6a1d6a9-2bfc-5a2e-a092-630ec990a896","task_key":"all--test--a6a1d6a9-2bfc-5a2e-a092-630ec990a896","task_revision_id":"1","upstream_id":"","short_description":"A law bookstore entered into a written contract to purchase from the publisher…","config":"all","split":"test","body":"{\"choices\":\"['enforceable, because it was not supported by any new consideration.', 'enforceable, under the principle of promissory estoppel.', 'unenforceable, because it is violative of the statute of frauds.', \\\"unenforceable, because the error resulted from the publisher's computational error.\\\"]\",\"question\":\"A law bookstore entered into a written contract to purchase from the publisher 100 copies of the latest edition of a certain casebook for $10 per book. Three days after the contract was formed, but prior to delivery of the casebooks, the publisher called the owner of the law bookstore and informed him that, because of a calculation error, the price for the casebooks should have been $11 per book, and the shipment could not be delivered unless the owner promised to pay that amount. The owner reluctantly agreed. The owner's agreement to pay $11 per book is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}