{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"331fda28-b058-5935-a5cf-16c578a7a719","task_key":"all--test--331fda28-b058-5935-a5cf-16c578a7a719","task_revision_id":"1","upstream_id":"","short_description":"A truck driver was making deliveries for his employer. After telling his…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because of the doctrine of vicarious liability.', 'Yes, because the employer owned the truck.', 'No, because the employee was not within the scope of his employment.', 'No, because liability cannot be imputed from one person to another.']\",\"question\":\"A truck driver was making deliveries for his employer. After telling his employer by phone that he was going for lunch and getting approval, he pulled into the entrance of a restaurant where he intended to have his usual 30- minute lunch break. While entering the parking lot, the driver carelessly ran over and seriously injured a pedestrian. The injured person sued the driver and the driver's employer for negligence in causing her injuries. Is the employer liable?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}