{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"736eefd4-8ff2-5ad3-8795-c2cc9a05ae8c","task_key":"all--test--736eefd4-8ff2-5ad3-8795-c2cc9a05ae8c","task_revision_id":"1","upstream_id":"","short_description":"A defendant has been charged with selling cocaine to a police informant. At…","config":"all","split":"test","body":"{\"choices\":\"['No, because identification of a controlled substance requires an expert with formal training.', 'No, because, without a quantity of the controlled substance for testing, opinion testimony is insufficient to make a prima facie case against the defendant.', 'Yes, if the court determines that the informant has sufficient knowledge and experience to identify cocaine.', 'Yes, provided there is evidence sufficient to support a jury finding that the informant has sufficient knowledge and experience to identify cocaine.']\",\"question\":\"A defendant has been charged with selling cocaine to a police informant. At trial, the alleged cocaine no longer exists, and the only evidence that the substance sold was cocaine is the informant's testimony that it tasted like cocaine and gave her a cocaine-like sensation. The informant has no formal training in identifying controlled substances. Should the court admit the informant's opinion testimony that the substance was cocaine?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}