{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7726afea-7bd1-5295-ba58-7b11fc568039","task_key":"all--test--7726afea-7bd1-5295-ba58-7b11fc568039","task_revision_id":"1","upstream_id":"","short_description":"A defendant decided to rob a bar because he knew that the bar was now utilizing…","config":"all","split":"test","body":"{\"choices\":\"['Arson, burglary, and larceny.', 'Arson and burglary.', 'Larceny and burglary.', 'Arson, robbery, and burglary.']\",\"question\":\"A defendant decided to rob a bar because he knew that the bar was now utilizing a new security system that was faulty and that nighttime was the best time to break in. Late one night, the defendant broke into the bar and proceeded to load large kegs of beer onto his truck. As he was leaving, the defendant inadvertently threw his cigarette butt into a trash can behind the bar. Soon afterward, the trash ignited into a small fire. Although the defendant had time to douse the fire without any danger to himself, he did not do so. Instead, he climbed into the truck and drove away. Although the fire spread beyond the trashcan, the fire department responded quickly and was able to put it out. The floor behind the bar was charred. The statutes for both burglary and arson in this jurisdiction include residential and business structures. The defendant should be convicted of which of the following crimes?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}