{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1652c10-217b-5fc9-868f-346f6fc68384","task_key":"all--test--c1652c10-217b-5fc9-868f-346f6fc68384","task_revision_id":"1","upstream_id":"","short_description":"A statute in the state makes it a misdemeanor for any motor vehicle to travel to…","config":"all","split":"test","body":"{\"choices\":\"['recover the fair market value of his house before the fire.', 'recover that part of his loss that would have been prevented if the businesswoman had not hit the fire truck.', 'recover nothing, because the businesswoman was not responsible for causing the fire.', 'recover nothing, because the traffic statute was not designed to protect against the type of harm that the owner suffered.']\",\"question\":\"A statute in the state makes it a misdemeanor for any motor vehicle to travel to the left of the center line of any two-way highway, road, or street. Late for a business appointment, a businesswoman was driving north on a highway when she decided to pass the car in front of her. As she swung across the center line into the southbound lane, her vehicle collided with a fire engine. As a result of the accident, the fire engine was delayed in reaching an owner's house, which was entirely destroyed by fire. The owner's home was located approximately one mile from the accident scene. If the owner asserts a claim against thebusinesswoman, he will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}