{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1e005d17-8d45-522e-9a74-9a2aa07473af","task_key":"default--test--5596","task_revision_id":"1","upstream_id":"5596","short_description":"Mulholland (1998) argued that privatization changed the relationship between…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the relationship between companies and managers became more formalized', 'ownership was transferred from small shareholders to senior managers', 'the state had greater control than managers over production processes', 'managers became less accountable to shareholders', 'privatization led to an increase in the number of small shareholders', 'privatization led to a decrease in the number of employees', 'flexibility was reduced by the introduction of detailed daily worksheets', 'employment depended on performance rather than trust and commitment', 'there was a shift in power from the managers to the employees', 'the role of managers remained unchanged despite privatization']\",\"question\":\"Mulholland (1998) argued that privatization changed the relationship between companies and managers in that:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}