{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"95293ab0-62fc-5a9c-9b5d-ac1ec805de9b","task_key":"all--test--95293ab0-62fc-5a9c-9b5d-ac1ec805de9b","task_revision_id":"1","upstream_id":"","short_description":"When the production or consumption of a good creates a positive externality, it…","config":"all","split":"test","body":"{\"choices\":\"['the marginal social benefit exceeds the marginal social cost.', 'the marginal social cost exceeds the marginal social benefit.', 'society produces too much of the good.', 'the private benefits from consuming the good exceed the social benefits.']\",\"question\":\"When the production or consumption of a good creates a positive externality, it is deemed a market failure because at the market quantity\",\"subject\":\"high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}