{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f8161590-b620-5967-9c46-2f7833a81d9b","task_key":"all--test--f8161590-b620-5967-9c46-2f7833a81d9b","task_revision_id":"1","upstream_id":"","short_description":"After months of negotiations, the United States and Canada entered into a tax…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The President's unilateral termination of a treaty benefiting residents of the United States cannot be effective until a hearing is afforded to persons who would be affected by such action.\\\", 'The courts have exclusive authority to determine whether a particular treaty has, in fact, been breached by another nation.', 'A ratified treaty is the supreme law of the land and, therefore, remains effective until superseded by another treaty or statute.', 'The treaty created a property right in Canadian citizens residing in the United States that cannot be taken away without just compensation.']\",\"question\":\"After months of negotiations, the United States and Canada entered into a tax treaty that provided that neither country would impose income taxes on citizens of the other nation. The treaty, which was ratified by the Senate, was supported by professional baseball and hockey players. Many Canadian hockey players, who were employed by American teams and lived in the United States during the hockey season, lobbied for passage of the treaty, since they were subject to the payment of both U. S. and Canadian income taxes. In like manner, many American baseball players, who lived in Canada during the baseball season, objected to the same dual taxation. In violation of the treaty, a Canadian court convicted a U. S. citizen who resided in Canada, of illegally evading the payment of Canadian income taxes. The U. S. citizen, who was a member of a Canadian baseball team, resided in Canada during the baseball season. Following his conviction, the President announced that the previously effective tax treaty would no longer be abided by the United States. The President proclaimed that Canada's refusal to honor the treaty rendered it invalid. As a result, the President ordered the Internal Revenue Service to begin collecting income taxes from Canadian citizens residing in the United States in the same manner that it collects taxes from other residents of this country. A Canadian citizen and resident of the United States, sues in an appropriate federal court, seeking a declaratory judgment that the treaty with Canada remains valid and effective. Therefore, he contends that the Internal Revenue Service may not collect U. S. income taxes from him. Which of the following is the strongest constitutional basis that may be urged in support of this claim?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}