{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"669973bc-1060-5adc-9921-63e3881d16bc","task_key":"default--test--991","task_revision_id":"1","upstream_id":"991","short_description":"A suspected drug dealer was arrested after selling three grams of cocaine to…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['both indictments.', 'the federal indictment, if the state indictment is dismissed.', 'both indictments, if the appeal is successful.', 'the state indictment, if the federal court conviction is upheld.', 'neither of the indictments.', 'the federal indictment, but not the state indictment.', 'the state indictment, but not the federal indictment.', 'the federal indictment, if the state court conviction is upheld.', 'both indictments, if the appeal is unsuccessful.', 'the state indictment, if the federal indictment is dismissed.']\",\"question\":\"A suspected drug dealer was arrested after selling three grams of cocaine to undercover police agents. He was subsequently convicted in state court of possession with intent to distribute narcotics. Following his conviction, the drug dealer was sentenced to a prison term of 10 years. After being sentenced, the drug dealer's attorney appealed the conviction, citing jury misconduct. While his appeal was pending, the drug dealer was then indicted by both a federal and a state grand jury for conspiracy to distribute the same cocaine. The drug dealer's attorney filed motions to dismiss each of the new indictments on the grounds that they violate double jeopardy. In all likelihood, double jeopardy requires the dismissal of\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}