{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39cfda1e-9249-5a7f-9138-c8223ba602d3","task_key":"default--test--7606","task_revision_id":"1","upstream_id":"7606","short_description":"Francois Quesnay and thePhysiocratsargued twenty years before Adam Smith that…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Wealth accumulation is solely based on financial investment and the stock market', 'Wealth originates from the labor of the working class', 'Wealth is derived from the arts and cultural industries', 'Wealth comes from commerce', 'Wealth is generated by government policy and regulation', 'Industry is the source of wealth', 'Natural resources other than agriculture, such as mining, are the true sources of wealth', 'Agriculture is the only possible source of new wealth', 'Wealth is a result of technological innovation and development', 'Wealth is created through trade']\",\"question\":\"Francois Quesnay and thePhysiocratsargued twenty years before Adam Smith that the source of all wealth is agri-culture . Why\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}