{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32e28b83-0db5-5539-b875-9b8441d65e2e","task_key":"all--test--32e28b83-0db5-5539-b875-9b8441d65e2e","task_revision_id":"1","upstream_id":"","short_description":"A defendant was arrested and charged with arson, which in this jurisdiction is…","config":"all","split":"test","body":"{\"choices\":\"['proper, because it constituted a rebuttable presumption of fact.', 'proper, because it constituted a permissible inference.', 'improper, because the prosecution must prove each element of the crime beyond a reasonable doubt.', 'improper, because the instruction shifts the burden of persuasion from the prosecution to the defendant defense.']\",\"question\":\"A defendant was arrested and charged with arson, which in this jurisdiction is defined as the \\\"malicious or intentional burning of any structure or dwelling. \\\" When the defendant was arrested, he was inside a warehouse that was engulfed in flames. At the time he was apprehended, the police found in his possession a book of matches with four matches missing. At trial, the prosecution introduced the book of matches into evidence. Following closing arguments, and before the jury retired for deliberations, the judge gave instructions concerning the law of the case. The judge instructed the jury that it could infer the defendant's intent to set the fire based on the fact that four matches were missing from the matchbook. This instruction was specifically objected to by the defense counsel. The judge's jury instruction concerning the defendant's intent was\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}