{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0a1dbf0-b67b-575c-aad3-8022efe14982","task_key":"all--test--c0a1dbf0-b67b-575c-aad3-8022efe14982","task_revision_id":"1","upstream_id":"","short_description":"A defendant charged with homicide had a long and tortured history of mental…","config":"all","split":"test","body":"{\"choices\":\"['Yes, the administration of medications to render the defendant capable of standing trial, without serious side effects, is an important governmental interest, and even more so in a homicide case.', 'Yes, the individual has no right to be protected from involuntary medical treatment when he is in custody of federal authorities.', 'No, the individual has an absolute constitutional right to be free from intrusion of medications into his system against his will.', 'No, the involuntary administration of medications to a criminal defendant is considered to be cruel and unusual punishment in violation of state and federal constitutional protections.']\",\"question\":\"A defendant charged with homicide had a long and tortured history of mental illness. He related several outrageous paranoid fantasies to the authorities and was sent to a mental hospital for evaluation of his ability to stand trial. The doctors reported that the only hope to restore the defendant to relative normalcy where he could stand trial was to administer widely-used anti-psychotic medicine. The man was also a danger to himself and others, and the medicine was in his medical best interests. The defendant asserted that the administration of drugs against his will would violate his liberty interest to substantive due process. Will the court likely order that the defendant should be involuntarily administered the anti-psychotic medications?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}