{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f6b3a51-d4de-504b-aacb-df1dad032d25","task_key":"all--test--8f6b3a51-d4de-504b-aacb-df1dad032d25","task_revision_id":"1","upstream_id":"","short_description":"A traffic accident occurred at a road intersection. A motorcycle, a car, and a…","config":"all","split":"test","body":"{\"choices\":\"['recover nothing, because he was more at fault than the driver of the truck.', 'recover $27,000.', 'recover $20,000.', 'recover $15,000.']\",\"question\":\"A traffic accident occurred at a road intersection. A motorcycle, a car, and a truck were involved. The motorcyclist was injured and brought suit against the driver of the car and the driver of the truck. The jury returned a verdict finding that the motorcyclist's injuries were caused by negligence on the part of all three of the parties. The jury assigned 55 percent of the fault for the motorcyclist's injuries to the motorcyclist, 25 percent to the driver of the car, and 20 percent to the driver of the truck. The jury found that the amount of the motorcyclist's injuries was$100,000. The motorcyclist enforces the judgment against the driver of the car and collects $45,000 from the driver of the car. If the driver of the car then brings an action against the driver of the truck for contribution, the driver of the car should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}