{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c2ddad9f-684d-5c48-b712-cee62e154524","task_key":"default--test--5428","task_revision_id":"1","upstream_id":"5428","short_description":"The opinions of congressional committees often fail to accurately mirror public…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['committees are often influenced by external factors such as media and public relations', 'committee members are not required to consult their constituents before making decisions', 'committees often prioritize national interests over public opinion', 'committee members are usually selected from the same political party', 'most committees are too small to include representatives of all popular opinions', 'the majority of committees are controlled by the executive branch', 'the minority party controls nearly half of all committees', 'the public does not have direct influence on the decision-making process of the committees', \\\"the membership of committees often overrepresents constituencies with the greatest stake in the committee's business\\\", \\\"committee members often lose track of their constituents' priorities because of their frequent contact with special-interest lobbyists\\\"]\",\"question\":\"The opinions of congressional committees often fail to accurately mirror public opinion because\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}