{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72ac5566-7cac-5493-a1b6-85d1835d2dc5","task_key":"default--test--1964","task_revision_id":"1","upstream_id":"1964","short_description":"Congress enacts a statute that makes Pennsylvania Avenue a one-way street.…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['It is invalid because it conflicts with the ordinance passed by the Washington, D.C., City Council.', \\\"It is valid as it is within the purview of Congress's power over transportation.\\\", 'It is invalid because the law infringes upon the rights of the residents of Washington, D.C.', 'It is invalid because it is a discriminatory burden on interstate commerce.', \\\"It is valid because it is a proper exercise of Congress's enumerated property power.\\\", 'It is invalid because it violates the Tenth Amendment.', 'It is valid because Congress has exclusive power over the District of Columbia.', 'It is valid because Congress has the power to regulate interstate commerce.', 'It is invalid because the Washington, D. C. , City Council has exclusive power over the public thoroughfares within the District of Columbia.', 'It is valid because it serves a clear governmental interest.']\",\"question\":\"Congress enacts a statute that makes Pennsylvania Avenue a one-way street. Congress has enacted the statute solely for the purpose of enabling members of Congress to travel back and forth to the Capital without being tied up in heavy traffic each day. Before Congress enacted this law, the Washington, D. C. , City Council had previously passed an ordinance designating Pennsylvania Avenue as a two-lane street. Which of the following is the most accurate statement regarding the constitutionality of the federal statute designating Pennsylvania Avenue a one-way street?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}