{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67afd42c-be99-50e8-8ddb-ecffdc526c0d","task_key":"default--test--5204","task_revision_id":"1","upstream_id":"5204","short_description":"Incumbent members of the House of Representatives win reelection more often than…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['representatives are more likely to secure party endorsements', 'senators are more likely to face strong challengers from within their own party', 'senators have more political power than representatives, and Senate races are accordingly contested more aggressively', 'representatives more often run uncontested', \\\"representatives' constituents more often belong largely to the representative's party\\\", 'senators have a larger geographic area to campaign in, making it more challenging', 'representatives may use federal funds to publicize their achievements via direct mail to constituents', 'representatives have shorter terms, allowing for more frequent engagement with constituents', 'senators have less visibility and recognition among voters', 'senators are more prone to controversies due to their high-profile positions']\",\"question\":\"Incumbent members of the House of Representatives win reelection more often than incumbent senators for all of the following reasons EXCEPT\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}