{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6cc17896-b271-5867-bc52-c5b0b0ba9db3","task_key":"all--test--6cc17896-b271-5867-bc52-c5b0b0ba9db3","task_revision_id":"1","upstream_id":"","short_description":"A furniture store had a truck that an employee of the store used to deliver…","config":"all","split":"test","body":"{\"choices\":\"['nothing, because the furniture store was primarily liable for the entire amount of damages.', \\\"$12,500, because the employer and employee bear equal shares of responsibility for the plaintiff's injuries.\\\", '$25,000, because the employee was at fault in causing the accident.', '$25,000, unless the furniture store was fully insured against such losses.']\",\"question\":\"A furniture store had a truck that an employee of the store used to deliver furniture to the store's customers. One afternoon while the employee was driving the truck to deliver a chair to a customer, the employee negligently ran through a stop sign and hit a car. The driver of the car suffered a leg injury and succeeded in suing the furniture store, recovering $25,000. Assume that all of these events occur in a jurisdiction that has statutes permitting defendants to bring claims for contribution and indemnity. In the event that the furniture store brings suit against its employee (the truck driver) to recover for its losses, the furniture store will recover\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}