{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b6e3db8-7347-53eb-b1da-e407854c576f","task_key":"default--test--5944","task_revision_id":"1","upstream_id":"5944","short_description":"There was a growth in income inequality in the 1980s because:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['inflation rate was higher than the growth rate of wages', 'the price of consumer goods rose at a higher rate than earnings', 'the government introduced policies favouring the rich', 'rates of income tax increased equally for all occupational groups', 'the cost of living decreased while wages remained the same', 'growing unemployment made more people dependent on welfare benefits', 'there was a surge in the number of low-paying jobs', 'there were more professional jobs available but not enough people to fill them', 'there was a decrease in the number of high-paying jobs', 'the minimum wage was significantly lowered']\",\"question\":\"There was a growth in income inequality in the 1980s because:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}