{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"accdb4ab-14aa-5d65-90ca-87145338e74d","task_key":"all--test--accdb4ab-14aa-5d65-90ca-87145338e74d","task_revision_id":"1","upstream_id":"","short_description":"A consumer bought a lawn mower from a department store. It has a prominent…","config":"all","split":"test","body":"{\"choices\":\"['No, because contributory negligence is not a defense to strict liability in a products liability case.', 'No, because the product was defective and unreasonably dangerous.', 'Yes, because the consumer assumed the risk of the danger when he used the product on a steep hill.', 'Yes, because the consumer ignored the warning and misused the product.']\",\"question\":\"A consumer bought a lawn mower from a department store. It has a prominent notice on it that it should not be used on a steep hill. The consumer did not read the notice and used the machine on a steep hill. As a result, it falls over and crushes him. He sues on a theory of strict liability, alleging a defect in the manufacture of the product. The manufacturer defends that the consumer misused the product. Will the Court likely grant the defendant's motion for summary judgment based on the indisputable fact that the consumer did not read the warning, which the defendant argued renders the lawsuit invalid as a matter of law?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}