{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f37c5d9-29cb-5161-b75f-c8fab2e895a5","task_key":"all--test--8f37c5d9-29cb-5161-b75f-c8fab2e895a5","task_revision_id":"1","upstream_id":"","short_description":"At a defendant's trial for drug dealing, a prosecution witness testified that he…","config":"all","split":"test","body":"{\"choices\":\"['No, because an expert must base his or her opinion on scientific or technical data, not merely on experience.', 'No, because the opinion is based on the criminal actions of others engaged in drug dealing, and thus is substantially more unfairly prejudicial than probative.', 'Yes, because the opinion is based on her specialized knowledge, and it will assist the trier of fact in understanding the evidence and determining facts in issue in the case.', 'Yes, provided that the expert first testifies about the specific information upon which she based her view that the defendant was using \\\"apples\\\" as a code word to refer to drugs.']\",\"question\":\"At a defendant's trial for drug dealing, a prosecution witness testified that he had heard the defendant telling a group of people to come to the defendant's house because he had \\\"a great crop of apples for sale.\\\" The prosecutor then called and qualified a drug enforcement agent as an expert in how drug dealing is conducted. The prosecutor now seeks to have the expert testify that in her opinion, based on years of experience with drug dealers, the defendant's statement about the apples was code for his having drugs for sale. Should the expert's opinion be admitted?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}