{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"afc2f61c-c97a-531d-a9b5-aa9636108749","task_key":"all--test--afc2f61c-c97a-531d-a9b5-aa9636108749","task_revision_id":"1","upstream_id":"","short_description":"Federal law prohibits \"willingly and knowingly\" taking cash in excess of $10,000…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because willfulness clause requires proof of both knowledge of the law and a specific intent to commit the crime.', 'Yes, because treaties with Canada make all such reporting laws unenforceable.', 'No, the practice is so dangerous to the public interest that knowledge and specific intent are not required.', 'No, willfulness and knowledge are inferred by the habitual practice of transporting the cash.']\",\"question\":\"Federal law prohibits \\\"willingly and knowingly\\\" taking cash in excess of $10,000 from the U.S. into a foreign country without first reporting the transaction in detail. An owner of a Detroit drug store takes his gross cash receipts each week into a city in Canada where he lives and does his banking. The office of the Deputy Atty. General learned that the owner was doing this, and indicted him on 10 counts of \\\"willingly and knowingly\\\" taking cash over $10,000 into a foreign country without reporting it. The owner's main defense is that he did not know of the law or that he was breaking it. The trial judge instructed the jury that mistake of law is no defense. He was convicted and appealed. Will the federal appellate court likely reverse the conviction?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}