{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ce4a84e-f47c-5b4f-b0a8-94d70a22b6a1","task_key":"all--test--1ce4a84e-f47c-5b4f-b0a8-94d70a22b6a1","task_revision_id":"1","upstream_id":"","short_description":"On March 1, a homeowner and a painter entered into a written contract wherein…","config":"all","split":"test","body":"{\"choices\":\"['No, because the homeowner is estopped to deny the validity of the written contract.', 'No, because the agreement regarding the loan varied the express terms of the writing.', 'Yes, because the agreement regarding the loan constituted a valid modification of the writing.', 'Yes, because the loan agreement was a condition precedent to the existence of the contract.']\",\"question\":\"On March 1, a homeowner and a painter entered into a written contract wherein the painter promised to paint the exterior of the homeowner's house for $3,000. Prior to the signing of the contract, the homeowner and the painter orally agreed that their contract would be null and void unless the homeowner was able to obtain a $3,000 loan from National Bank before April 1. On March 31, the homeowner was informed by National Bank that his loan application had been rejected. The next day, the homeowner telephoned the painter and informed him that the deal was off. If the painter brings an action for breach of contract against the homeowner, would the latter's inability to secure the loan provide him with a valid defense?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}