{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01f8803f-f2cf-5bf1-8185-c68e09800be3","task_key":"all--test--01f8803f-f2cf-5bf1-8185-c68e09800be3","task_revision_id":"1","upstream_id":"","short_description":"A property owner held title in fee simple to a tract of 20 acres located outside…","config":"all","split":"test","body":"{\"choices\":\"['prevail, unless the person who was responsible for damaging her car can be identified.', 'prevail, if the damage to her car could have been prevented had the property owner taken reasonable security measures.', 'not prevail, because the car was damaged by the malicious acts of an independent third person.', 'not prevail, because the local police had the primary duty to provide security protection at the shopping area.']\",\"question\":\"A property owner held title in fee simple to a tract of 20 acres located outside the boundaries of the city. Thereafter, the property owner constructed a shopping center on the property and leased commercial buildings and parking facilities to various tenants. The shopping center, which was located near a public high school, attracted many teenagers who often loitered in the parking lot. The youths frequently harassed shoppers and damaged autos by breaking off windshield wipers and radio antennas. Customarily, the local police department patrolled the shopping center and drove by three or four times each day. This, however, did not prevent the teenagers from hanging out at the shopping center. One afternoon, a shopper was shopping at the center when an unidentified youth damaged her car by throwing a rock through the back window. The shopper brings an action against the property owner to recover for the damage to her auto. She will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}