{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4df2339-30cf-5532-9c8f-0fb1d01041be","task_key":"all--test--c4df2339-30cf-5532-9c8f-0fb1d01041be","task_revision_id":"1","upstream_id":"","short_description":"A buyer purchased a new dishwasher from an appliance store. This dishwasher had…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the dishwasher was defective.', 'Yes, because she was the purchaser of the dishwasher.', 'No, because the buyer was not personally injured.', 'No, because a reasonably prudent person would have discontinued using the dishwasher after becoming aware of the rattling noise.']\",\"question\":\"A buyer purchased a new dishwasher from an appliance store. This dishwasher had been manufactured by an appliance company. After the dishwasher had been installed in the buyer's home, it functioned properly for the first month. Then it began to make a loud rattling noise whenever it was in use. The buyer called the appliance store, and a manager there assured her that it was not uncommon for the dishwasher to make such a noise. Moreover, he indicated that the machine's clamorous sound would eventually disappear with continued use. One week later, after a dinner party, the buyer placed her most expensive china set in the dishwasher and turned on the machine. After the dishwasher had been in operation for a few minutes, the buyer heard the rattling noise followed by the sound of breaking china. She immediately stopped the machine. Upon looking inside, she saw that a blade from the dishwasher had broken, destroying her entire china set. If the buyer asserts a claim for strict products liability against the appliance company, will she prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}