{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d3097c3b-9022-5a2b-8442-9a187ecb960b","task_key":"all--test--d3097c3b-9022-5a2b-8442-9a187ecb960b","task_revision_id":"1","upstream_id":"","short_description":"The president of a pharmaceutical firm received a report from his testing bureau…","config":"all","split":"test","body":"{\"choices\":\"['No, because the intentional delivery of adulterated or mislabeled drugs gives rise to strict criminal liability.', 'No, because the jury could have found that the presidentÕs conduct was sufficiently reckless to constitute murder.', 'Yes, because distribution of the defective lot was only a regulatory offense.', 'Yes, because the cancer, not the presidentÕs conduct, was the proximate cause of death of the patient.']\",\"question\":\"The president of a pharmaceutical firm received a report from his testing bureau that a manufactured lot of the firmÕs anticancer prescription medication was well below strength. Concerned about being able to fulfill contractual commitments, the president instructed his staff to deliver the defective lot. A cancer patient who had been maintained on the drug died shortly after beginning to take the defective pills. Medical evidence established that the patient would have lived longer had the drug been at full strength, but would have died before long in any event. The president was convicted of murder. On appeal, he argues that his conviction should be reversed. Should the conviction be reversed?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}