{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e31f75a-91a4-5a90-8028-2836dffc2973","task_key":"default--test--7215","task_revision_id":"1","upstream_id":"7215","short_description":"Private property, freedom of choice and the pricing mechanism are some features…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Advanced technology, extensive specialization, use of money', 'Central planning, equal income distribution, state ownership of resources', 'Government-regulated monopolies, planned obsolescence, compulsory consumption', 'Market competition, free trade, consumerism', 'Communistic ideology, socialistic policies, barter system', 'Nationalized industries, welfare state, progressive taxation', 'Universal basic income, cooperative business models, digital currency adoption', 'Private property, freedom of choice, pricing mechanism', 'Collective farming, absence of currency, shared ownership of goods', 'Public goods provision, mixed economic policies, environmental sustainability']\",\"question\":\"Private property, freedom of choice and the pricing mechanism are some features that are peculiar to capitalism. These characteristics alone, however, do not accurately describe modern economies. Identify three essential features of modern economies that are not peculiar to capitalism.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}