{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01c682bb-6cca-525e-ba3a-c6cd5ba155a5","task_key":"all--test--01c682bb-6cca-525e-ba3a-c6cd5ba155a5","task_revision_id":"1","upstream_id":"","short_description":"A consumer purchased a new station wagon from a car dealer. The car dealer…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the brakes failed while the consumer was driving his car.', 'Yes, because the brakes failed because of a defect present when the consumer purchased the car.', 'No, because the consumer contributed to his own injury by speeding up.', 'No, because the car dealer carefully inspected the car before selling it.']\",\"question\":\"A consumer purchased a new station wagon from a car dealer. The car dealer performed a thorough inspection of the station wagon before the sale was completed. One evening, the consumer was driving along a city street when he saw the traffic light facing him turn from green to amber. He sped up, hoping to cross the intersection before the light turned red. However, he quickly realized that he could not do so and applied the brakes, which failed because of a defect in the brake mechanism. The consumer then swerved to avoid hitting a bus that was crossing the intersection at a right angle to him. As a result of the swerve, the consumer's car rode up on the sidewalk and overturned, pinning a pedestrian. Both the consumer and the pedestrian were severely injured. If the consumer asserts a claim against the car dealer based on strict liability in tort, will the plaintiff prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}