{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5d2d2eed-0e50-5cd4-8a5b-ccb583de57a2","task_key":"default--test--1725","task_revision_id":"1","upstream_id":"1725","short_description":"Congress recently enacted a statute permitting a governmental agency to make a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The Secretary of Defense himself.', 'The national organization.', 'A federal court.', 'A taxpayer of the United States.', 'A member of the church.', 'A member of Congress.', 'Any resident in the United States.', 'A citizen of the United States.', 'A foreign government.', 'A state within the United States.']\",\"question\":\"Congress recently enacted a statute permitting a governmental agency to make a gift of federal property to private individuals or organizations, provided that it be used \\\"to further good relations and better understanding with foreign governments. \\\"The Secretary of Defense planned to give an old military cargo plane to a national organization who supports future pilots. Before making the gift, the Secretary was approached by the head of a church. The church leader indicated that he would be sponsoring a worldwide crusade and suggested to the Secretary that such an undertaking would serve to strengthen relations with foreign governments. The Secretary donated the plane to the church instead of the organization. Who would have the best standing to challenge the Secretary's action?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}