{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd786817-916e-5e7f-9443-b055363ee1e1","task_key":"default--test--7242","task_revision_id":"1","upstream_id":"7242","short_description":"If a monopsony labor market suddenly were transformed into a perfectly…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The wage would decrease, but employment would remain constant.', 'Both would increase.', 'Both would decrease.', 'The wage would remain constant, but employment would increase.', 'The wage would increase, but employment would remain constant.', 'The wage would increase, but employment would decrease.', 'Employment would decrease, but the wage would increase.', 'Employment would increase, but the wage would decrease.', 'The wage would fall, but employment would increase.', 'Both would remain constant.']\",\"question\":\"If a monopsony labor market suddenly were transformed into a perfectly competitive labor market, how would the wage and employment change?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}