{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0522cab4-dc43-5d9f-ad4b-edd0998b1ad4","task_key":"all--test--0522cab4-dc43-5d9f-ad4b-edd0998b1ad4","task_revision_id":"1","upstream_id":"","short_description":"After being passed over for a promotion, an aeronautic engineer became a…","config":"all","split":"test","body":"{\"choices\":\"['admissible as former testimony.', 'admissible as a vicarious admission.', 'inadmissible as hearsay not within any recognized exception.', 'inadmissible, because the company did not have the opportunity to cross-examine the engineer on the liability issue for which the statement is now being offered.']\",\"question\":\"After being passed over for a promotion, an aeronautic engineer became a whistleblower. He contacted a government agency to state that the company for which he worked was submitting false safety reports on newly built aircraft. When the company learned that the engineer was leaking this information to the government agency, he was fired from his job. Afterward, the engineer sued the company for wrongful termination of employment. During the discovery stage of litigation, the engineer was deposed by the company's attorney. In his deposition, the engineer stated that the company submitted false safety reports to the government agency to cover up structural defects in its aircraft. A pilot was injured when one of the company'sairplanes he was piloting crashed. The pilot asserteda strict products liability tort against the company. At thal, the pilot seeks to introduce into evidence portions of the engineer's deposition from his wrongful termination lawsuit against the company. Assume that the engineer is unavailable to testify at trial. Upon objection by the company, the trial court judge should rule the engineer's deposition testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}