{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99ae570d-95ff-5113-affb-c1020476a5b0","task_key":"all--test--99ae570d-95ff-5113-affb-c1020476a5b0","task_revision_id":"1","upstream_id":"","short_description":"A homeowner offered to pay a roofer he knew $2,000 for the roofer's agreement to…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because an offer for a bilateral contract can be revoked before its acceptance.', \\\"No, because the act of purchasing supplies specifically for the homeowner's job was an effective acceptance and a contract was formed prior to homeowner's attempted revocation.\\\", 'No, because the offer had to remain open for a reasonable period of time before it could be revoked.', 'Yes, because an oral offer to enter a bilateral contract is never enforceable.']\",\"question\":\"A homeowner offered to pay a roofer he knew $2,000 for the roofer's agreement to make repairs to the homeowner's roof. He said to the roofer, \\\"We'll have a deal when you begin to do the work necessary for this job.\\\" The roofer did not directly respond to the offer but he went out and purchased the supplies and materials needed for the homeowner's specifications. When he arrived at the site with the supplies he found a message from the homeowner that he revoked the offer. Can the homeowner revoke the offer under these circumstances?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}