{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80607799-64ad-56c2-8b33-d8718899038f","task_key":"default--test--6885","task_revision_id":"1","upstream_id":"6885","short_description":"Suppose consumers expect the price of corn to be lower in the near future while…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The equilibrium price will rise while the equilibrium quantity is indeterminate.', 'The equilibrium price will fall while the equilibrium quantity is indeterminate.', 'The equilibrium price is indeterminate while the equilibrium quantity will rise.', 'The equilibrium price is indeterminate while the equilibrium quantity will fall.']\",\"question\":\"Suppose consumers expect the price of corn to be lower in the near future while the price of seed corn and fertilizer fall. What will happen to the equilibrium price and the equilibrium quantity?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}