{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"116a6c23-bae6-5081-8d81-493de9a0c9c1","task_key":"all--test--116a6c23-bae6-5081-8d81-493de9a0c9c1","task_revision_id":"1","upstream_id":"","short_description":"In an automobile accident case, the defendant claimed that the plaintiff was…","config":"all","split":"test","body":"{\"choices\":\"['overrule the objection, because the judge may take judicial notice of the speed limit within the jurisdiction.', 'overrule the objection if the witness has already been shown to be an experienced driver who was able to estimate the approximate speed of moving vehicles.', 'sustain the objection if the witness has not been qualified as an expert witness.', 'sustain the objection, because the witness is not an adverse witness.']\",\"question\":\"In an automobile accident case, the defendant claimed that the plaintiff was contributorily negligent by driving at an excessive speed. The defendant's attorney calls its first witness, who was standing along the side of the road when the plaintiff's and defendant's cars collided. The defendant's attorney asks the witness: \\\"Is it a fact that in your estimation, plaintiff was traveling at a speed of 45 m. p. h. in a 35 m. p. h. speed zone at the time of the accident?\\\" The plaintiff objects. The trial judge should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}