{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b301fd51-53fc-5508-99ad-53f09a0afad5","task_key":"all--test--b301fd51-53fc-5508-99ad-53f09a0afad5","task_revision_id":"1","upstream_id":"","short_description":"Which of the following transactions correctly illustrates the doctrine of…","config":"all","split":"test","body":"{\"choices\":\"['Blair ordered a dozen blue chairs from Kyle but Kyle delivered a dozen red chairs.', 'Leslie painted an entire room but failed to put the electrical outlet covers back on.', \\\"A contract required hair styling to be done to Toby's satisfaction but Toby was in good faith dissatisfied with the completed result.\\\", \\\"A dentist competently completed the extraction of Lee's tooth but mistakenly pulled the wrong one.\\\"]\",\"question\":\"Which of the following transactions correctly illustrates the doctrine of substantial performance?\",\"subject\":\"professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}