{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"692f30f6-6bdf-5c53-8314-4893a755eb59","task_key":"all--test--692f30f6-6bdf-5c53-8314-4893a755eb59","task_revision_id":"1","upstream_id":"","short_description":"A witness lived next door to a victim. Late one night, the witness overheard the…","config":"all","split":"test","body":"{\"choices\":\"['admissible as a dying declaration if the jury determines that the victim believed that her death was imminent.', 'admissible as a dying declaration if the judge, by preponderance of the evidence, determines that the victim believed that her death was imminent.', 'inadmissible, because the probative value is substantially outweighed by the danger of unfair prejudice.', 'inadmissible as hearsay not within any recognized exception.']\",\"question\":\"A witness lived next door to a victim. Late one night, the witness overheard the victim scream, \\\"The defendant, please don't do it!\\\" The next day the victim was found dead in her bedroom. The defendant was arrested and charged with murdering the victim. At trial, the witness proposes to testify to the victim's statement. Upon objection by defendant's counsel, the court should rule the witness's testimony regarding the victim's statement\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}