{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2881ee3-390c-5e4c-aeab-4608fbaa18c2","task_key":"all--test--a2881ee3-390c-5e4c-aeab-4608fbaa18c2","task_revision_id":"1","upstream_id":"","short_description":"On May 1, a homeowner and a painter entered into a contract whereby the painter…","config":"all","split":"test","body":"{\"choices\":\"['The homeowner would not be obligated to pay the nephew, since the contract was nonassignable.', 'Since personal service contracts of this nature are nonassignable, the homeowner would be under no duty to pay the nephew.', 'The assignment would constitute a novation, relieving the homeowner of liability.', 'The assignment would be irrevocable if it were reduced to writing and signed by the painter.']\",\"question\":\"On May 1, a homeowner and a painter entered into a contract whereby the painter was to paint the homeowner's home in exchange for $10,000. The contract stated:\\\"The painting is to be done during the month of June, and payment is to be made within one week after the painting is completed. The rights under this contact are not to be assigned. \\\"The painter started the painting on June 1. On June 10, the painter assigned to his nephew the right to receive payment on the painter's contract with the homeowner. Which of the following statements is most accurate with regard to the painter's assignment to his nephew?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}