{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c238a15-7c14-53ba-a359-900c6a277fec","task_key":"all--test--3c238a15-7c14-53ba-a359-900c6a277fec","task_revision_id":"1","upstream_id":"","short_description":"A man and a woman were partners in an automobile leasing agency. For some time,…","config":"all","split":"test","body":"{\"choices\":\"['Conspiracy and murder.', 'Murder.', 'Solicitation and conspiracy.', 'Solicitation, conspiracy, and murder.']\",\"question\":\"A man and a woman were partners in an automobile leasing agency. For some time, the man had been embezzling money from the agency. The woman became suspicious and told the man that she wanted to engage an accounting firm to audit the agency's books. In a panic to prevent the audit and avoid being discovered, the man decided to have the woman killed. He contacted a professional killer and hired him to kill the woman for $10,000. A short time later, the woman died in a car crash arranged by the killer. In addition to embezzlement, of what crime or crimes may the man properly be convicted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}