{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aae8fc85-d6c0-5e4f-b9bb-7bb098f3b8f2","task_key":"all--test--aae8fc85-d6c0-5e4f-b9bb-7bb098f3b8f2","task_revision_id":"1","upstream_id":"","short_description":"A man was arrested and charged with sexually assaulting a child. At trial, in…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, because the man's past crimes may be introduced to show that he is more likely to have committed the crime with which he is charged.\\\", 'admissible, because a common plan or scheme may be proven by other similar criminal acts.', 'inadmissible, because past crimes may not be introduced to prove disposition to commit certain acts.', 'inadmissible, because the prosecution cannot initiate evidence of bad character.']\",\"question\":\"A man was arrested and charged with sexually assaulting a child. At trial, in its case-in-chief, the prosecution seeks to introduce evidence that the man had twice been previously involved in similar cases with other children. The prosecution argues that these past crimes demonstrate the likelihood that the man molested this child. Defense counsel objects to the evidence regarding other children. The judge should find the evidence\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}