{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47a8fb65-87f5-5cb7-b232-49566d33cf12","task_key":"default--test--7046","task_revision_id":"1","upstream_id":"7046","short_description":"Suppose a price floor is installed in the market for coffee. One result of this…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['a decrease in the price of coffee.', 'an increase in consumer surplus due to lower coffee prices.', 'a persistent shortage of coffee in the market.', 'a decrease in the demand for coffee.', 'a decrease in consumer surplus due to higher coffee prices.', 'a persistent surplus of coffee in the market.', 'an increase in the demand for coffee.', 'a decrease in the supply of coffee due to lower prices.', 'a decrease in the demand for coffee-brewing machines.', 'an increase in producer surplus due to higher coffee prices.']\",\"question\":\"Suppose a price floor is installed in the market for coffee. One result of this policy would be\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}