{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b7b5a3c-d0a4-56c7-9f73-6badaf008a17","task_key":"all--test--7b7b5a3c-d0a4-56c7-9f73-6badaf008a17","task_revision_id":"1","upstream_id":"","short_description":"The President appointed a delegation to enter into negotiations with…","config":"all","split":"test","body":"{\"choices\":\"['The presidential power to conduct foreign affairs.', 'An ancillary power of the President under his treaty-making power.', 'The treaty-making power, but only if the treaty is ratified by two-thirds of the Senate.', 'The treaty-making power, but only if the treaty is ratified by a majority in Congress.']\",\"question\":\"The President appointed a delegation to enter into negotiations with representatives of a foreign government to study the problem of preventing the extinction of certain species of seals. The delegation's goal was twofold: to study the problem and to formulate regulations in a bilateral agreement that would protect the endangered species and provide for a permanent commission that would continually monitor enforcement of the proposed regulations. After compiling their findings and drafting the necessary regulations, the President and the leader of the foreign government entered into a treaty to form a permanent commission to oversee the problem and to grant it the necessary enforcement powers. The validity of this treaty would most likely be upheld under which of the following principles?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}