{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7db4be83-a4ea-5bd0-a679-447dc05c8615","task_key":"all--test--7db4be83-a4ea-5bd0-a679-447dc05c8615","task_revision_id":"1","upstream_id":"","short_description":"A defendant, who was an indigent, was arrested and charged with possession of…","config":"all","split":"test","body":"{\"choices\":\"['A $500 fine.', 'Six months in jail.', 'One year in jail.', 'No sentence, because the defendant was denied the right to counsel.']\",\"question\":\"A defendant, who was an indigent, was arrested and charged with possession of less than one ounce of marijuana. Under the relevant state statute, possession of less than one ounce of marijuana is a misdemeanor, punishable by a maximum of one year in jail and/or a fine of $500. At trial, the defendant pleaded not guilty and requested that an attorney be appointed to represent him. The trial judge refused to honor the defendant's request for an attorney. If the defendant is subsequently found guilty, which of the following is the most severe sentence that can constitutionally be imposed?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}