{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c9fde0ac-33da-5d17-a85a-c7534a6d9fa9","task_key":"default--test--7499","task_revision_id":"1","upstream_id":"7499","short_description":"If consumers are advised that multigrained bread will substantially lessen the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The supply curve will shift to the right, decreasing the price of multigrained bread.', 'The demand curve will shift to the left, decreasing the price of multigrained bread.', 'The demand and supply curves will both shift to the left, keeping the price of multigrained bread stable.', 'The demand and supply curves will both shift to the right, keeping the price of multigrained bread stable.', 'The supply curve will shift to the left, decreasing the price of multigrained bread.', 'The demand curve will shift to the right, increasing the price of multigrained bread.', 'The supply curve will shift to the left, increasing the price of multigrained bread.', 'The supply curve will shift to the right, increasing the price of multigrained bread.', 'The demand and supply curves will not be affected, keeping the price of multigrained bread stable.', 'The demand curve will shift to the left, increasing the price of multigrained bread.']\",\"question\":\"If consumers are advised that multigrained bread will substantially lessen the risk of cancer, which of the following will happen in the market for multigrained bread?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}