{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6f09c8e-7213-54d0-98e2-6d29415d0b05","task_key":"all--test--d6f09c8e-7213-54d0-98e2-6d29415d0b05","task_revision_id":"1","upstream_id":"","short_description":"A construction company, in preparing its bid for the construction of a new…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because of the unilateral mistake.', 'prevail, because the mistake was an essential element of the bargain.', 'not prevail, unless the construction company knew or should have known of the subcontractor \\\\x80\\\\x98s error.', 'not prevail, because the computation mistake was antecedent to acceptance of the bid.']\",\"question\":\"A construction company, in preparing its bid for the construction of a new hospital, received a quotation of $120,000 from a subcontractor who offered to do the kitchen work in the new hospital. This bid was $30,000 lower than the construction company's next lowest bid for the kitchen work. As a result, the construction company lowered its bid by $20,000 before submitting it to the hospital board. After the construction company was awarded the construction bid and had accepted the subcontractor's offer, the subcontractor discovered that in his preparation of the quotation, he had overlooked some subsidiary kitchen installments required by the plans. Immediately thereafter, the subcontractor brought suit for rescission of the contract. It should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}