{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdc630bc-691d-5d40-addd-32944755e0f6","task_key":"all--test--bdc630bc-691d-5d40-addd-32944755e0f6","task_revision_id":"1","upstream_id":"","short_description":"A truck driver from State A and a bus driver from State B were involved in a…","config":"all","split":"test","body":"{\"choices\":\"['The court should apply the federal common law of negligence.', \\\"The court should apply the negligence law of State A, the truck driver's state of citizenship.\\\", 'The court should consider the negligence law of both State A and State B and apply the law that the court believes most appropriately governs negligence in this action.', \\\"The court should determine which state's negligence law a state court in State B would apply and apply that law in this action.\\\"]\",\"question\":\"A truck driver from State A and a bus driver from State B were involved in a collision in State B that injured the truck driver. The truck driver filed a federal diversity action in State B based on negligence, seeking $100,000 in damages from the bus driver. What law of negligence should the court apply?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}