{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dddf4d5e-df39-595a-8e86-6d62cb6e3ce3","task_key":"all--test--dddf4d5e-df39-595a-8e86-6d62cb6e3ce3","task_revision_id":"1","upstream_id":"","short_description":"A defendant is on trial for false pretenses. He is charged with selling…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, to show defendant's character trait for dishonesty.\\\", 'admissible, to show his intent to defraud.', 'inadmissible, because character cannot be proved by specific instances of misconduct.', 'inadmissible, because the evidence is not relevant.']\",\"question\":\"A defendant is on trial for false pretenses. He is charged with selling worthless stock in a dummy corporation to unwitting investors. The defendant is alleged to have masterminded a scheme wherein he set up a nonexistent corporation that never conducted business. The victims were sent prospectuses containing false financial data, which induced them to purchase stock in the phony corporation. At trial, the prosecution seeks to introduce into evidence proof that the defendant had set up 10 other so-called dummy corporations that never existed. This evidence is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}