{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f6f4f80-323b-52f6-80a0-00a20bebe5bd","task_key":"default--test--1845","task_revision_id":"1","upstream_id":"1845","short_description":"A state has had a tremendous influx of retired people in recent years. There has…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['unconstitutional, because it violates the commerce clause, since Congress has plenary power to regulate any activity that has any appreciable effect on interstate commerce.', 'unconstitutional, because it creates an unfair trade barrier for non-physicians wanting to start a business in the state.', 'constitutional, as it aims to protect a vulnerable segment of the population from potential exploitation.', 'constitutional, because there is a rational basis for distinguishing between non-physicians are not so engaged.', 'unconstitutional, because it discriminates between non-physicians already in the business and those wanting to enter it.', 'unconstitutional, because it restricts the rights of non-physicians to conduct a legitimate business.', \\\"constitutional, because it falls under the state's power to regulate healthcare and related services.\\\", 'constitutional, because it does not infringe upon the rights of non-physicians not already in the business of selling hearing aids.', 'constitutional, because a state has the power to regulate any phase of local business, even though such regulations may have some effect on interstate commerce, provided that Congress has not enacted legislation regarding the subject matter.']\",\"question\":\"A state has had a tremendous influx of retired people in recent years. There has been considerable concern among state health officials who foresee that many of the senior citizens will become victims of price gouging on certain medical supplies and services. In an attempt to curb such fraudulent sales practices, the state legislature has enacted a law prohibiting the sale of hearing aids by non-physicians. The measure provides, however, that all non-physician sellers who are presently engaged in the business of selling hearing aids will not be affected. Assume that after the statute goes into effect, a non-physician moves to the state and wants to open a business selling hearing aids. After being advised that the state law prohibits him from doing so, he brings suit challenging the constitutionality of the statute. The most likely result is that the state law will be declared\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}