{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5d6d2a8-7ecc-51c8-a610-7a6a32c54ad3","task_key":"default--test--7340","task_revision_id":"1","upstream_id":"7340","short_description":"Suppose that the trade-unions are strong enough to bargain a 5% increase in…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['2%', '0%, if the wage increase is completely offset by an equal increase in productivity.', '7%, assuming a significant gap between wage increase and productivity growth leading to higher inflation.', '5%', '4%, considering other economic factors that might contribute to a price increase.', '2.5%, accounting for a scenario where the wage increase slightly outpaces the combined effect of productivity growth and other inflationary factors.', '3%', '6%, assuming prices rise more than the wage increase due to other inflationary pressures.', '1%', 'Workers are not better off at all; prices rise by 5%, negating the wage increase.']\",\"question\":\"Suppose that the trade-unions are strong enough to bargain a 5% increase in nominal wages. How much do youexpect the prices will rise? In the U.S.A. productivity increase on average of 2% per year. How much are the workers better off ?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}