{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8af72e32-0a43-5fd3-81d9-3cc587032501","task_key":"default--test--5498","task_revision_id":"1","upstream_id":"5498","short_description":"Which of the following defines the salience of a political issue?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The amount of coverage the issue receives in the major news media', 'The number of social media posts about the issue', 'The geographical area impacted by the issue', 'The length of time the issue has been present', 'The number of politicians who have publicly spoken about the issue', 'The number of protests organized around the issue', 'The amount of funding allocated to the issue by the government', 'The degree to which public opinion on the issue is likely to shift quickly', 'The importance of the issue to a particular individual or group', 'The number of people affected by the issue']\",\"question\":\"Which of the following defines the salience of a political issue?\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}