{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8105af8c-c654-577a-b970-f63948130796","task_key":"default--test--1086","task_revision_id":"1","upstream_id":"1086","short_description":"Two brothers who were in need of money approached their wealthy uncle's…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Solicitation, conspiracy to commit murder of their uncle, and attempted murder of the physician.', 'Murder of both their uncle and the physician.', 'Conspiracy to commit murder of their uncle, and murder of the physician.', 'Solicitation, conspiracy to commit murder of their uncle, and involuntary manslaughter of the physician.', 'Conspiracy to commit murder of their uncle and the physician.', 'Conspiracy to commit murder of their uncle, and murder of the housekeeper.', 'Conspiracy to commit murder of their uncle, and attempted murder of the physician.', 'Solicitation, conspiracy to commit murder of their uncle, and murder of the physician.', 'Conspiracy to commit murder of their uncle only.', 'Conspiracy to commit murder of their uncle, and involuntary manslaughter of the physician.']\",\"question\":\"Two brothers who were in need of money approached their wealthy uncle's housekeeper and asked her to poison him. The brothers would inherit the uncle's estate when he died. The housekeeper agreed, but on the condition that they would pay her $10,000 from their inheritance. After the brothers agreed to her demand, the housekeeper decided to place some cyanide in the uncle's tea one morning. As the housekeeper was preparing the tea, the uncle was visited by his personal physician. When the housekeeper was ready to serve the tea, the uncle asked her to fix some tea for his physician also. The housekeeper did so and then carefully set the tea on the table so that the uncle's cup was facing him. However, when the physician reached over to get some sugar, he inadvertently took the uncle's cup with the cyanide and drank it. Seconds later, the physician died from the poison. Which of the following crimes are the brothers guilty of?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}