{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23668454-c0c6-52df-a6c9-87ce45a00b39","task_key":"default--test--7503","task_revision_id":"1","upstream_id":"7503","short_description":"A negative externality in the market for a good exists when","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['spillover benefits are received by society.', 'total welfare is maximized.', 'the marginal social benefit equals the marginal social cost.', 'the marginal private benefit equals the marginal social cost.', 'there are no spillover benefits or costs to society.', 'the market overallocates resources to the production of this good.', 'total welfare is minimized.', 'the market underallocates resources to the production of this good.', 'spillover costs are absorbed by the producer.']\",\"question\":\"A negative externality in the market for a good exists when\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}