{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"37f2958b-f45b-5ff0-9763-1d10fb211424","task_key":"all--test--37f2958b-f45b-5ff0-9763-1d10fb211424","task_revision_id":"1","upstream_id":"","short_description":"\"An assault is an unlawful attempt, with or without present ability, to commit a…","config":"all","split":"test","body":"{\"choices\":\"['guilty, because he intended to frighten the senior partner.', 'guilty, because the statute does not require the present ability to commit a battery.', 'not guilty, because the defendant did not intend to cause physical injury to the senior partner.', 'not guilty, because the facts state that the senior partner was not frightened.']\",\"question\":\"\\\"An assault is an unlawful attempt, with or without present ability, to commit a battery on the person of another. It is a misdemeanor punishable by imprisonment of up to six months. \\\"A defendant was a lawyer who worked in a law firm located in the state. The defendant, who had a reputation of being a prankster, decided to play a practical joke on the senior partner of the law firm. The defendant donned a cowboy mask and entered the senior partner's office. As the senior partner was talking on the phone with his back to the door, the defendant pulled out a toy cap gun and said, \\\"This is a stickup . . . give me your wallet or I'll shoot. \\\" When the senior partner turned around to face the defendant, he pulled the trigger of the gun. This caused the toy pistol to make a loud noise. Although the senior partner was not frightened, he fired the defendant because of the incident, citing unprofessional conduct. The defendant was subsequently charged with criminal assault under the above mentioned statute. At trial, the defendant testified that he was only playing a practical joke on the senior partner and was trying to scare him as a harmless prank. If the jury believes the defendant, they should find him\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}