{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9f6b349-a005-5285-82b8-21040a05374e","task_key":"default--test--5869","task_revision_id":"1","upstream_id":"5869","short_description":"The alternative form of 'network organization' means that:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the company operates solely through social media platforms', 'the company does not have a physical office, all tasks are performed online', 'each department works independently with no interaction with other departments', 'work can be subcontracted out to independent suppliers and retailers', 'all employees work remotely from different locations', 'business transactions occur only through electronic communication', 'the company has multiple branch offices in different cities or countries', 'there is no hierarchy, everyone is at the same level', 'the Japanese model is applied, through lateral networks of flexible roles', 'activities are redistributed equally between men and women']\",\"question\":\"The alternative form of 'network organization' means that:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}