{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3358cd9b-7e4c-5543-b127-9889b0b62d4b","task_key":"default--test--7626","task_revision_id":"1","upstream_id":"7626","short_description":"Suppose DVD players are a normal good and are exchanged in a competitive market.…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Have no impact on the equilibrium quantity or price.', 'Have no impact on the equilibrium quantity but increase the price.', 'Have no impact on the equilibrium price but increase the quantity.', 'decrease the equilibrium quantity and increase the price.', 'increase the equilibrium price but the change in quantity is ambiguous.', 'Decrease the equilibrium price but the change in quantity is ambiguous.', 'increase the equilibrium quantity and increase the price.', 'decrease the equilibrium quantity and decrease the price.', 'Increase the equilibrium quantity but decrease the price.', 'Increase the equilibrium price but decrease the quantity.']\",\"question\":\"Suppose DVD players are a normal good and are exchanged in a competitive market. All else equal an increase in household income will\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}