{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"edbeb5fd-5b64-56d9-ae5e-20949de0895b","task_key":"default--test--981","task_revision_id":"1","upstream_id":"981","short_description":"A state built a racetrack that was specially designed and constructed for…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['unconstitutional, because it denies citizens under the age of 65 the equal protection of the law.', 'constitutional, because it supports social welfare by providing discounts to senior citizens.', 'unconstitutional, because it discriminates on the basis of age.', 'constitutional, because the state has the power to change admission prices.', 'unconstitutional, because it violates the rights of the bondholders.', \\\"unconstitutional, because it goes against the state's responsibility to repay bonds.\\\", 'constitutional, because the state has the right to regulate commerce within its borders.', 'unconstitutional, because it is an ex post facto law.', \\\"unconstitutional, because it impairs the obligation of the bondholders' contract.\\\", \\\"constitutional, because it is a justifiable exercise of the state's police power.\\\"]\",\"question\":\"A state built a racetrack that was specially designed and constructed for thoroughbred horseracing. State bonds were issued to finance the construction of the racetrack. The bond agreement provided that for the first five years the racetrack was in operation, at least $2 from each admission charge would be used to repay the bond debt. The bond agreement further stipulated that if the proceeds from the admission charges during the first five years were not sufficient to repay the bondholders, then at least $1 from each admission charge for the next five years would be applied to make the necessary bond payments. After the racetrack was built and in operation for two years, the state legislature passed a bill requiring the racetrack to admit all senior citizens over the age of 65 for the discounted admission fee of 50 cents. This law is probably\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}