{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b8cd82d3-56da-5edd-8acb-0dba1c5ea745","task_key":"all--test--b8cd82d3-56da-5edd-8acb-0dba1c5ea745","task_revision_id":"1","upstream_id":"","short_description":"A defendant has been charged with manufacturing illegal firearms in violation of…","config":"all","split":"test","body":"{\"choices\":\"[\\\"as a business record, because the custodian can testifS' as to how the records of the college were kept.\\\", 'under the general hearsay exception, as the transcript in these circumstances has sufficient circumstantial guarantees of trustworthiness.', \\\"only if the registrar's signature can be authenticated by an expert witness.\\\", 'only if the defendant waives his right to object to its admission.']\",\"question\":\"A defendant has been charged with manufacturing illegal firearms in violation of federal firearms laws. In its case-in-chief, the government wants to introduce a copy of the defendant's transcript from a technical college to show his certificate as gunsmith. The college is no longer in existence, though, and the whereabouts of the previous officials and employees are unknown. However, the transcript bears a seal above the registrar's signature, and the court has taken judicial notice of the existence of the college and has found that it is normal for a college to compile a transcript. Finally, the information about the defendant on the transcript has been corroborated by a government witness. Assume the prosecution provided the defendants with a copy of the transcript before trial and informed them of its intent to introduce it as evidence. The transcript will be admitted\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}