{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9aadf458-0344-5f80-8681-c4cbd2da1ef8","task_key":"default--test--7571","task_revision_id":"1","upstream_id":"7571","short_description":"A competitive market for coffee, a normal good, is currently in equilibrium.…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The price of coffee machines rises.', 'The wage of coffee plantation workers falls.', 'The price of tea rises.', 'A decrease in the popularity of energy drinks.', 'Technology in the harvesting of coffee beans improves.', 'Consumer income falls.', 'A decrease in the price of milk.', 'An increase in the production of coffee beans.', 'The price of coffee beans falls.', 'An increase in the number of coffee shops.']\",\"question\":\"A competitive market for coffee, a normal good, is currently in equilibrium. Which of the following would most likely result in an increase in the demand for coffee?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}