{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc9ee99e-394c-5ed1-9c94-ca77cb530c41","task_key":"default--test--6904","task_revision_id":"1","upstream_id":"6904","short_description":"Which of the following is true about the pure public good?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The marginal cost of providing an additional unit of the good to additional citizens is zero.', 'The consumption of public goods leads to depletion of resources.', 'The public good can be consumed by only a limited number of consumers at a time.', 'One more unit of a public good for some consumers means one less unit for other consumers.', 'Consumers can be excluded from its benefits by not paying for it.', 'The benefits of public goods can be privatized by corporations.', 'The good cannot be divided into discrete units.', 'The marginal cost of providing an additional unit of the good to additional citizens is greater than zero.', 'Public goods can only be consumed by those who pay for them.', 'Public goods are always provided by government agencies.']\",\"question\":\"Which of the following is true about the pure public good?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}