{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f26ca71-721c-51be-b6a7-a897a09dcb73","task_key":"all--test--9f26ca71-721c-51be-b6a7-a897a09dcb73","task_revision_id":"1","upstream_id":"","short_description":"The state charged the accused with the intentional murder of a former…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the wording of the instruction could have been viewed by jurors as a mandatory direction to find that there was intent to murder.', 'Yes, because the wording in effect caused the defendant to be a witness against himself by having him intend everything that he did.', 'No, because the defendant could raise an argument to rebut the permissible inference created by the jury instruction.', 'No, because the instruction was only a presumption that the jury did not have to consider in its deliberations.']\",\"question\":\"The state charged the accused with the intentional murder of a former girlfriend. He admitted to killing her, but asserted that he lacked the intent to murder, due to his suffering from a personality disorder complicated by two days of drinking alcohol heavily. The trial judge instructed the jury that \\\"the law presumes that a person intends the ordinary consequences of his voluntary acts.\\\" The judge did not tell them that they had a choice or could infer another conclusion; they were told only that the law presumed it. The accused was convicted of depraved heart murder and sentenced to 100 years in prison. On appeal, the appellant argued that the court took the fact-finding task of determining criminal intent out of the jury's hands by in effect ordering them to find intent. The jury could have felt that they were being told that they had no other choice. The state supreme court affirmed the conviction, but the United States Supreme Court granted certiorari. Will the Court likely reverse the conviction?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}