{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6584f35-1317-5c57-8503-2958006cc0d8","task_key":"default--test--1308","task_revision_id":"1","upstream_id":"1308","short_description":"A man and a woman conspired together to rob a bank. As they were exiting the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, because the felony murder rule only applies when the defendant is the one who directly causes the death.', 'Yes, because the police officer who shot the woman should be the one charged with murder, not the man.', 'No, because felony murder applies regardless of who fires the fatal shot as long as a death occurred during the commission of the felony.', \\\"No, because the man's decision to raise his gun at the officers instigated the shooting, making him responsible for the woman's death.\\\", 'No, because all states recognize that felony murder applies to any deaths that occur during the commission of the crime regardless of who fired the fatal shot.', 'Yes, because the felony murder rule has been declared unconstitutional.', 'Yes, because the woman was a co-conspirator, not a victim of the crime, and therefore her death does not qualify as a felony murder.', \\\"No, because the man started the flurry of shots instead of surrendering; he caused the woman's death and is guilty of first degree murder.\\\", 'Yes, because the majority view is that felony murder does not apply if a co-conspirator is killed by a third person during the commission of the crime.', \\\"No, because the man was part of the crime and his actions led directly to the woman's death.\\\"]\",\"question\":\"A man and a woman conspired together to rob a bank. As they were exiting the bank after the robbery they were confronted by armed tactical police officers. The man raised his gun at the officers and a flurry of shots followed. The man was injured but survived; the woman co-conspirator died from a police officer's bullet. The authorities charged the man with felony-murder of the woman. He was convicted and appealed. Under the prevailing modern view of felony murder, will the appellate court likely reverse the first-degree murder conviction?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}