{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f26f2f2-3455-5cb6-88b8-811a1e8c7d69","task_key":"default--test--631","task_revision_id":"1","upstream_id":"631","short_description":"In the 'five forces model', what is the type of competitive activity that exists…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['The threat of industry regulation', 'The potential for mergers and acquisitions', 'The level of market saturation', 'The power of suppliers', 'The level of rivalry', 'The threat of substitute products', 'The potential for entry into the industry', 'The power of shareholders', 'The potential for innovation', 'The power of customers']\",\"question\":\"In the 'five forces model', what is the type of competitive activity that exists between organisations?\",\"src\":\"ori_mmlu-management\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}