{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f65627d0-f824-5e7b-896f-720ccb0c0017","task_key":"default--test--5733","task_revision_id":"1","upstream_id":"5733","short_description":"The primary function of political action committees (PACs) is to","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['determine the platforms of political parties', 'conduct independent investigations into political corruption', 'provide legal advice to candidates for election', 'promote the defeat of incumbents in the federal and state legislatures', 'provide candidates with campaign staff and resources', 'organize protest demonstrations and other acts of civil disobedience', 'coordinate local get-out-the-vote campaigns', 'directly campaign for specific candidates', 'lobby for legislation in Congress', 'contribute money to candidates for election']\",\"question\":\"The primary function of political action committees (PACs) is to\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}