{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"972c2d8a-4054-5225-8cb5-85a9ef7d21b7","task_key":"all--test--972c2d8a-4054-5225-8cb5-85a9ef7d21b7","task_revision_id":"1","upstream_id":"","short_description":"A used car dealer needs to disclaim any implied warranty of merchantability.…","config":"all","split":"test","body":"{\"choices\":\"['By stating in the contract that there is no \\\"implied warranty of merchantability.', 'By stating in writing that the vehicle is sold \\\"as is.\\\"', 'By setting forth an express disclaimer that is clear, on the point, and conspicuous.', 'By limiting the amount to the value of the vehicle sold.']\",\"question\":\"A used car dealer needs to disclaim any implied warranty of merchantability. Which of the following is not considered an effective way of making a full disclaimer of the implied warranty?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}