{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"008535bf-066e-510b-b892-6050fd54a1bc","task_key":"default--test--5065","task_revision_id":"1","upstream_id":"5065","short_description":"According to The Federalist Papers, federalism has which of the following…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['It eliminates any opportunity for factions to form.', 'It allows factions to dominate state governments while limiting their influence on the national level.', 'It results in a two-party system, limiting the number of factions.', 'It encourages the formation of more factions to create a balance of power.', 'It guarantees equal representation for all factions in the government.', 'It limits the dangers of factionalism by diluting political power.', 'It completely suppresses the formation of factions.', 'It promotes a single dominant faction at the national level.', 'It allows factions to dominate on the national level while limiting their influence on state governments.', 'It provides a structured environment in which factions may flourish.']\",\"question\":\"According to The Federalist Papers, federalism has which of the following effects on political factions?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}