{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da9ed871-720a-5a8a-8cf5-03d39d5f764d","task_key":"all--test--da9ed871-720a-5a8a-8cf5-03d39d5f764d","task_revision_id":"1","upstream_id":"","short_description":"A mechanic agreed in writing to make repairs to a landscaper's truck for…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the landscaper had no reasonable alternative but to yield to the mechanic's wrongful threat.\\\", 'No, because the mechanic exerted undue influence over the landscaper with respect to the modification.', 'Yes, because the landscaper could have obtained possession of the truck through legal action rather than by agreeing to the increased payment.', 'Yes, because the modified contract price did not exceed a reasonable price.']\",\"question\":\"A mechanic agreed in writing to make repairs to a landscaper's truck for $12,000. The mechanic properly made the repairs, but when the landscaper tendered payment, the mechanic refused to deliver the truck unless the landscaper promised to pay an additional $2,000. The customary charge for such work was $14,000. Because the landscaper needed the truck immediately to fulfill existing contractual obligations, and because no rental trucks of the same type were available, the landscaper promised in writing to pay the mechanic an additional $2,000. The mechanic then delivered the truck. Will the mechanic be able to enforce the landscaper's promise to pay the additional $2,000?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}