{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48b3467c-02be-5e84-b163-c3a0aa22ea71","task_key":"all--test--48b3467c-02be-5e84-b163-c3a0aa22ea71","task_revision_id":"1","upstream_id":"","short_description":"At a defendant's trial for the murder of his former employer, the prosecutor…","config":"all","split":"test","body":"{\"choices\":\"[\\\"sustain the objection on the grounds that the former girlfriend's testimony would be inadmissible opinion evidence.\\\", 'sustain the objection on the grounds that identification of handwriting requires expert testimony.', 'overrule the objection on the grounds that the former girlfriend qualifies as an authenticating witness.', 'overrule the objection on the grounds that the letter qualifies as a past recollection recorded exception to the hearsay rule.']\",\"question\":\"At a defendant's trial for the murder of his former employer, the prosecutor offers into evidence a note that was found during a lawful search of the defendant's apartment. The note states: \\\"The demon compels me to kill my former employer. I thirst for his bloody spirit. \\\" At trial, the defendant's former girlfriend is called by the prosecution to testify that the handwriting was, in fact, the defendant's. The defendant's attorney objects. The trial judge should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}