{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02f7741b-e55c-5831-908c-78035279f9e3","task_key":"all--test--02f7741b-e55c-5831-908c-78035279f9e3","task_revision_id":"1","upstream_id":"","short_description":"A relevant fact in a plaintiff's suit is the magnitude of an explosion at an oil…","config":"all","split":"test","body":"{\"choices\":\"['admissible, because the son is available as a witness.', 'admissible, whether or not the son is available as a witness.', 'inadmissible as hearsay not within any recognized exception.', 'inadmissible under the excited utterance exception, because it can be a product of reflection and deliberation.']\",\"question\":\"A relevant fact in a plaintiff's suit is the magnitude of an explosion at an oil refinery one evening. At trial, the plaintiff was asked on direct examination if he remembered the explosion. He replied, \\\"I recall my son running into the home and screaming that the car windows just shattered. \\\" The son was available to testify at trial. The defendant's attorney makes a motion to strike the plaintiff's answer. The trial judge should rule the plaintiff's testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}