{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40482712-de05-54c9-9c41-7ebf7674dca3","task_key":"all--test--40482712-de05-54c9-9c41-7ebf7674dca3","task_revision_id":"1","upstream_id":"","short_description":"A state has the following statute in effect:\"No person shall sell, barter,…","config":"all","split":"test","body":"{\"choices\":\"[\\\"result in her acquittal, because she didn't have the necessary mental state required for the crime.\\\", 'result in her acquittal, because the friend is 16 years of age or older.', 'not prevent her conviction, because mistake of law is no defense.', 'not prevent her conviction, because mistake of fact is no defense.']\",\"question\":\"A state has the following statute in effect:\\\"No person shall sell, barter, furnish, or give to a minor under 16 years of age an air gun, rifle, shotgun, pistol, or other firearm; or being the owner or having charge or control thereof, knowingly permit it to be used by a minor under such age. Whoever violates this statute shall be fined not more than $1,500 (one thousand five hundred dollars), or imprisoned not more than 45 days, or both. \\\"A mother purchased an air rifle, which she gave to her son. The mother, a police officer, who was familiar with firearms, trained her son in the systematic practice of care in the use of the air rifle. One afternoon, the son, who was 15 years of age, was playing with his friends. The three boys took turns firing the air rifle, which discharged small pellets, at various targets in the son's back yard. As one of the friends, who was 16 years old, was using the air rifle, he fired a shot over the other friend's head, intending to frighten him. The pellet missed the other friend, but struck a next-door neighbor in the eye, severely injuring her. The mother is subsequently charged with violating the state statute. As her defense, the mother claims that she erroneously believed the statute prohibited firearms to be given or sold to minors under 15 years of age. If the mother's mistaken belief is honest, it should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}