{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3934629f-08cc-5577-bec6-1b3b5dd7aacb","task_key":"default--test--7056","task_revision_id":"1","upstream_id":"7056","short_description":"When the production or consumption of a good creates a positive externality, it…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['society produces too little of the good.', 'the social benefits from consuming the good exceed the private benefits.', 'the marginal social benefit exceeds the marginal social cost.', 'the marginal private benefit exceeds the marginal social cost.', 'the marginal social cost exceeds the marginal social benefit.', 'the private costs from consuming the good exceed the social costs.', 'the private benefits from producing the good exceed the social benefits.', 'the private benefits from consuming the good exceed the social benefits.', 'the marginal private cost exceeds the marginal social benefit.', 'society produces too much of the good.']\",\"question\":\"When the production or consumption of a good creates a positive externality, it is deemed a market failure because at the market quantity\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}