{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"de7823f9-6e0c-5d4c-a240-4ae50d1f38a1","task_key":"all--test--de7823f9-6e0c-5d4c-a240-4ae50d1f38a1","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued the manufacturer of a slicing machine for negligent design,…","config":"all","split":"test","body":"{\"choices\":\"['No, because the change in design may have been unrelated to this type of accident.', 'No, under the rule regarding remedial measures that encourages manufacturers to make their products safer.', 'Yes, as evidence tending to show that the machine could be designed to keep the wires from coming into contact.', 'Yes, as evidence tending to show that the manufacturer was negligent because its initial design failed to prevent the wires from coming into contact.']\",\"question\":\"A plaintiff sued the manufacturer of a slicing machine for negligent design, after the machine cut off the plaintiff's finger. The plaintiff claims that while he was cleaning the machine, two wires came into contact with each other and caused the machine to turn on. At trial, the manufacturer has offered evidence that it was unreasonably expensive to design the machine so that the wires could not come into contact. In rebuttal, the plaintiff offers evidence that after this action was filed, the manufacturer redesigned the machine to prevent the wires from redesigned the machine to prevent the wires from coming into contact. Is evidence of this change in design admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}