{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d258258b-8728-50da-98f0-87842155b24f","task_key":"all--test--d258258b-8728-50da-98f0-87842155b24f","task_revision_id":"1","upstream_id":"","short_description":"A defendant is on trial for possession of heroin. During the prosecution's…","config":"all","split":"test","body":"{\"choices\":\"['No, because the chemist lacks firsthand knowledge that the powder came from the defendant.', 'No, unless the envelope and powder are produced in court or their absence is explained.', 'Yes, because an object in official custody is self-authenticating.', 'Yes, because the powder is sufficiently identified as having come from the defendant.']\",\"question\":\"A defendant is on trial for possession of heroin. During the prosecution's case-in-chief, a police officer testifies that he watched another officer seize a bag of white powder from the defendant and seal it in an envelope, which both officers initialed and dated and then placed in the police property room. The prosecutor then calls a chemist to testify that she obtained, intact from the property room, the dated, sealed envelope with the initials of both police officers, whose handwriting she recognized police officers, whose handwriting she recognized from previous experience, and that testing disclosed that the powder therein was heroin. The defendant argues that the prosecutor has not sufficiently authenticated the powder tested by the chemist as the powder that was seized from the defendant. Is the chemist's testimony admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}