{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ed2cdb0-6c82-5398-8efd-3d6965281f28","task_key":"all--test--4ed2cdb0-6c82-5398-8efd-3d6965281f28","task_revision_id":"1","upstream_id":"","short_description":"A commissioner of a sports league was charged with embezzling funds by…","config":"all","split":"test","body":"{\"choices\":\"['admissible, to show that the commissioner lacked criminal intent.', 'admissible as evidence of the routine practice of an organization.', 'inadmissible as hearsay not within any recognized exception.', \\\"inadmissible, because the league's charter is the best evidence.\\\"]\",\"question\":\"A commissioner of a sports league was charged with embezzling funds by increasing the amount of his allocated travel expenses without the approval of a majority of the team owners. In accordance with the bylaws of the league's charter, the commissioner was required to obtain the approval of at least half the owners before raising his expense allocation. At trial, the commissioner seeks to testify that his predecessor told him that he had authority under the league's charter to raise his expense allocation. The commissioner's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}