{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ce85f20-0b18-53b2-89b5-6b3b7f9638f5","task_key":"default--test--7010","task_revision_id":"1","upstream_id":"7010","short_description":"What is meant by consumers' surplus?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"[\\\"Consumers' surplus is the tax imposed on consumers by the government\\\", \\\"Consumers' surplus is the extra cost consumers have to pay due to high demand\\\", \\\"Consumers' surplus is the amount of goods consumers receive for free as part of a promotion\\\", \\\"Consumers' surplus is the savings consumers accumulate over time by choosing cheaper alternatives\\\", \\\"Consumers' surplus is the surplus of value consumers receive from all prior units, which have higher (marginal) utilities than the last unit bought.\\\", \\\"Consumers' surplus is the additional utility gained by consumers from purchasing luxury items\\\", \\\"Consumers' surplus is the discount offered by sellers during a sale period\\\", \\\"Consumers' surplus is the additional profit made by sellers\\\"]\",\"question\":\"What is meant by consumers' surplus?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}