{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81036320-c2b0-5260-a8ed-71018bcbab63","task_key":"default--test--5654","task_revision_id":"1","upstream_id":"5654","short_description":"Which of the following is true of court cases in which one private party is…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['They are always resolved through arbitration, not court trials.', 'They are always tried in the Supreme Court.', 'They are tried in juvenile court.', 'They are tried in international court.', 'The federal court system has exclusive jurisdiction over them.', 'The state court system has exclusive jurisdiction over them.', 'They are exclusively tried in appellate court.', 'The military court system has exclusive jurisdiction over them.', 'They are tried in civil court.', 'They are tried in criminal court.']\",\"question\":\"Which of the following is true of court cases in which one private party is suing another?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}