{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68a9bf17-49c1-5ce1-8c0e-311a500a772a","task_key":"all--test--68a9bf17-49c1-5ce1-8c0e-311a500a772a","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued the manufacturer of his lawn mower, alleging that, as the…","config":"all","split":"test","body":"{\"choices\":\"['No, because it goes to an ultimate issue that only the jury can decide.', 'No, because the manufacturer paid the expert to render a certain opinion, in violation of rules barring paid testimony. violation of rules barring paid testimony.', 'Yes, because expert testimony on such issues of causation is relevant and helpful to the jury.', 'Yes, provided that the plaintiff had notice and an opportunity to participate in the testing process.']\",\"question\":\"A plaintiff sued the manufacturer of his lawn mower, alleging that, as the result of a design defect, he was injured when the mower's blade flew off after striking a sprinkler head in the lawn. At trial, the manufacturer has called as an expert witness a product safety engineer, who testifies that the manufacturer retained him for a fee to test identical mowers and, if his opinion was helpful, to testify in the case. The expert then testifies that he did test the mowers, and that the blade, as designed and installed by the manufacturer, could not fly off in the manner claimed by the plaintiff. Assume that the expert has used a reliable method for reaching his conclusion. Should the court admit the expert's testimony?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}