{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a970b513-881f-5478-8f3e-4cf929839bb5","task_key":"all--test--a970b513-881f-5478-8f3e-4cf929839bb5","task_revision_id":"1","upstream_id":"","short_description":"A drug to combat high cholesterol was approved by the FDA and marketed by its…","config":"all","split":"test","body":"{\"choices\":\"['Yes, strict liability applies to pharmaceutical companies and questions of knowledge of defects is irrelevant.', 'Yes, the manufacturer must assume the cost of compensating those innocent victims who took the defective drug.', \\\"No, the company has no duty to conduct research to determine a product's safety.\\\", 'No, when the medical or scientific information regarding a defect has not yet been discovered, the company will not be held strictly liable for putting that product on the market.']\",\"question\":\"A drug to combat high cholesterol was approved by the FDA and marketed by its manufacturer. Reports of the drug being tied to cases of premature dementia surfaced, and a number of lawsuits were filed by injured consumers, alleging strict liability for a defective product. At trial, the manufacturer proved that thorough testing was done but that new scientific discoveries, not then available, now show a vulnerability in certain people to the drug. Is the company strictly liable to the victims even though the medical knowledge was not available when the product was developed and put on the market?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}