{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35a508c1-8ee6-56c7-a843-8633fb514f25","task_key":"all--test--35a508c1-8ee6-56c7-a843-8633fb514f25","task_revision_id":"1","upstream_id":"","short_description":"A vehicle operator was driving on a highway at night. His girlfriend was in the…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the passenger was not actively participating in a joint enterprise and did not do enough to contribute to the cause of the accident.', \\\"Yes, because a vehicle operator is always liable and a passenger cannot be made liable for the driver's actions under any circumstances.\\\", 'No, because the driver has properly alleged a joint enterprise situation where the passenger is liable for her share of the negligence in causing the accident.', 'No, because the passenger was breaking the law, making her negligent per se.']\",\"question\":\"A vehicle operator was driving on a highway at night. His girlfriend was in the passenger seat. They were returning from a college night class. She was smoking marijuana, blowing it out the open window, and not sharing it with the driver. The driver fell asleep, causing the car to veer off the highway and go into a ditch where it landed upside-down. Both the driver and passenger were seriously injured. She sued the operator for negligence. The operator filed a counterclaim asking for damages against the passenger by arguing that the passenger was herself negligent by \\\"smoking weed\\\" and causing him to fall asleep. Is the court likely to grant the passenger's motion to dismiss the counterclaim?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}