{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38819513-1c6f-5275-ab3c-75de33148ba5","task_key":"default--test--939","task_revision_id":"1","upstream_id":"939","short_description":"The President appointed a delegation to enter into negotiations with…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The treaty-making power, but only if the treaty is ratified by a majority of the states.', 'The treaty-making power, but only if the treaty is ratified by a majority in Congress.', 'The presidential power to conduct foreign affairs.', 'The presidential power to create commissions.', 'The treaty-making power, but only if the treaty is ratified by the House of Representatives.', 'The treaty-making power, but only if the treaty is approved by the Supreme Court.', 'The treaty-making power, but only if the treaty is ratified by two-thirds of the Senate.', 'The presidential power to enforce environmental regulations.', 'An ancillary power of the President under his treaty-making power.', 'The treaty-making power, but only if the treaty is ratified by unanimous vote in Congress.']\",\"question\":\"The President appointed a delegation to enter into negotiations with representatives of a foreign government to study the problem of preventing the extinction of certain species of seals. The delegation's goal was twofold: to study the problem and to formulate regulations in a bilateral agreement that would protect the endangered species and provide for a permanent commission that would continually monitor enforcement of the proposed regulations. After compiling their findings and drafting the necessary regulations, the President and the leader of the foreign government entered into a treaty to form a permanent commission to oversee the problem and to grant it the necessary enforcement powers. The validity of this treaty would most likely be upheld under which of the following principles?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}