{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cff3dd2c-3dbe-5c6b-b193-f83e401db98e","task_key":"default--test--7452","task_revision_id":"1","upstream_id":"7452","short_description":"What are the chief characteristics of the British brandof socialism?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Compulsory military service, government-controlled media, and a one-party political system', 'Strict adherence to Marxist principles, state ownership of all personal property, and the abolition of private enterprise', 'Evolutionary rather than revolutionary, government ownership and direction of key industries, extensive government planning of land use and housing, a nationalized health service, heavy redistributive income and estate taxes, and a great proliferation of social welfare agencies.', 'Focus on individual entrepreneurship, minimal welfare provision, and a strong emphasis on traditional values', 'Privatization of health services', 'Free market and low taxes', 'Decentralized planning, emphasis on mutual aid societies, and minimal government intervention', 'Revolutionary and violent', 'A mixed economy with partial privatization of health services, moderate taxation, and targeted social welfare programs', \\\"Emphasis on green policies, extensive use of referenda for policy decisions, and a citizens' basic income\\\"]\",\"question\":\"What are the chief characteristics of the British brandof socialism?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}