{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88d5f897-57b5-59b5-9164-6168e48b96a5","task_key":"default--test--5206","task_revision_id":"1","upstream_id":"5206","short_description":"Which of the following is a significant difference between the Senate and the…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The Senate allows unlimited debate, while the House does not.', 'Only the House has the power to issue subpoenas.', 'Only the Senate has the power to declare war.', 'States are equally represented in the House, while representation in the Senate is proportional to population.', 'Only the Senate may introduce revenue bills.', 'The Senate has more members than the House.', 'Only the House can propose amendments to the Constitution.', 'Only the House can confirm or reject presidential appointments.', 'The Senate has a term of six years, while the House has a term of four years.']\",\"question\":\"Which of the following is a significant difference between the Senate and the House of Representatives?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}