{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"217fb110-0908-5a70-b233-c739b2662520","task_key":"default--test--1564","task_revision_id":"1","upstream_id":"1564","short_description":"A security guard relocated to a state which required all persons wishing to…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The security guard has the burden of persuasion to show that the state law is discriminatory and violates his equal protection rights.', 'The state has the burden of persuasion to show that the licensing statute is essential for public safety and welfare.', 'The security guard has the burden of persuasion to show that the denial of a license to work in private security violates his rights of due process.', 'The security guard has the burden of persuasion to show that the state does not have a compelling state interest in enacting such legislation.', 'The state has the burden of persuasion to prove that the security guard does not have sufficient skills and training to work as a private security guard in the state.', \\\"The state has the burden of persuasion to show that the denial of a license to work in private security does not violate the petitioner's rights of due process.\\\", 'The state has the burden of persuasion to prove that the licensing statute is not a form of economic protectionism.', \\\"The state has the burden of persuasion to show that the licensing statute does not infringe on the security guard's right to work.\\\", 'The state has the burden of persuasion to show a compelling state interest in enacting such legislation.', 'The security guard has the burden of persuasion to show that the state law is excessively punitive and constitutes cruel and unusual punishment.']\",\"question\":\"A security guard relocated to a state which required all persons wishing to practice private security to have graduated from a security training program within the state. The security guard, having already graduated from a program in his former state, wishes to work again as a private security guard. He began to advertise his services but was informed by the state that he must cease and desist from working as a private security guard until he complied with state law and graduated from an in-state program. The security guard challenges the constitutionality of the licensing statute on grounds that it violates the due process clause of the Fourteenth Amendment. Which of the following statements is most accurate?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}