{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ae5f959-1a78-59a6-8000-ed5439f32b6e","task_key":"default--test--307","task_revision_id":"1","upstream_id":"307","short_description":"The enforcement of company privacy is complex as corporations are somewhat…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Public spaces, Control information, Public spaces', 'Boundaryless, Maintain individual privacy, Private spaces', 'Public spaces, Maintain individual privacy, Private spaces', 'Boundaryless, Control information, Private spaces', 'Private spaces, Control information, Public spaces', 'Private spaces, Maintain individual privacy, Public spaces', 'Boundaryless, Maintain individual privacy, Public spaces', 'Public spaces, Control information, Private spaces', 'Boundaryless, Control information, Public spaces', 'Private spaces, Maintain individual privacy, Private spaces']\",\"question\":\"The enforcement of company privacy is complex as corporations are somewhat __________, as there are few clear barriers to define their limits, it is difficult to _________ due to the number of individuals involved and much corporate activity takes place in ________.\",\"src\":\"ori_mmlu-business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}