{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01eea063-62d6-55a9-8024-aface85be850","task_key":"default--test--933","task_revision_id":"1","upstream_id":"933","short_description":"A defendant decided to rob a bar because he knew that the bar was now utilizing…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Robbery and burglary.', 'Only Arson.', 'Only burglary.', 'Arson and larceny.', 'Arson, larceny, and robbery.', 'Larceny and burglary.', 'Arson, robbery, burglary, and larceny.', 'Arson, burglary, and larceny.', 'Arson and burglary.', 'Arson, robbery, and burglary.']\",\"question\":\"A defendant decided to rob a bar because he knew that the bar was now utilizing a new security system that was faulty and that nighttime was the best time to break in. Late one night, the defendant broke into the bar and proceeded to load large kegs of beer onto his truck. As he was leaving, the defendant inadvertently threw his cigarette butt into a trash can behind the bar. Soon afterward, the trash ignited into a small fire. Although the defendant had time to douse the fire without any danger to himself, he did not do so. Instead, he climbed into the truck and drove away. Although the fire spread beyond the trashcan, the fire department responded quickly and was able to put it out. The floor behind the bar was charred. The statutes for both burglary and arson in this jurisdiction include residential and business structures. The defendant should be convicted of which of the following crimes?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}