{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7e70f11-595d-55b3-adf0-e87a2e0aa83a","task_key":"all--test--c7e70f11-595d-55b3-adf0-e87a2e0aa83a","task_revision_id":"1","upstream_id":"","short_description":"Congress recently enacted a statute permitting a governmental agency to make a…","config":"all","split":"test","body":"{\"choices\":\"['A citizen of the United States.', 'A taxpayer of the United States.', 'The national organization.', 'A state within the United States.']\",\"question\":\"Congress recently enacted a statute permitting a governmental agency to make a gift of federal property to private individuals or organizations, provided that it be used \\\"to further good relations and better understanding with foreign governments. \\\"The Secretary of Defense planned to give an old military cargo plane to a national organization who supports future pilots. Before making the gift, the Secretary was approached by the head of a church. The church leader indicated that he would be sponsoring a worldwide crusade and suggested to the Secretary that such an undertaking would serve to strengthen relations with foreign governments. The Secretary donated the plane to the church instead of the organization. Who would have the best standing to challenge the Secretary's action?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}