{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1636cd25-8c3d-5108-aee7-300429f54a09","task_key":"default--test--5797","task_revision_id":"1","upstream_id":"5797","short_description":"Given the structure of the electoral college system, presidential candidates…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['focus their campaigns only on the West Coast states', 'spend more television advertising money in the Plains States', 'spend most of their campaign budget on online advertising', 'campaign equally in all states regardless of their political leaning', 'avoid battleground states and focus their campaigns on \\\"safe\\\" states', 'campaign more heavily in states with large populations', 'ignore the battleground states and focus on states with a clear party preference', 'campaign extensively in states that heavily favor one party over the other']\",\"question\":\"Given the structure of the electoral college system, presidential candidates tend to\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}