{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f28b8234-76cf-52c7-9797-4ce52339c661","task_key":"default--test--7677","task_revision_id":"1","upstream_id":"7677","short_description":"Is an increase in national income equivalent to an increase in employment?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['An increase in national income has no effect on employment.', 'An increase in national income leads to an increase in employment when it is driven by technology advancements.', 'Yes, every increase in national income always results in an increase of employment.', 'An increase in national income results in an increase in employment only in the manufacturing sector.', 'No, every increase in national income does not necessarily result in an increase of employment.', 'An increase in national income always results in a decrease in employment.', 'An increase in national income is only related to employment in the service sector.', 'An increase in national income will result in increased employment if it is accompanied by equivalent productivity growth.', 'An increase in national income correlates with employment rates, but other factors such as labor market policies are more influential.', 'An increase in national income results in an increase in employment only when it exceeds inflation.']\",\"question\":\"Is an increase in national income equivalent to an increase in employment?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}