{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0cffe81-2f09-59ed-aaed-fb1d24aff5f8","task_key":"default--test--5907","task_revision_id":"1","upstream_id":"5907","short_description":"In an e-commerce environment that requires that the information technology (IT)…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The need for physical security measures to protect the IT system', 'Increased need for public relation strategies to manage potential IT failures', 'Maintain redundant systems for instant availability to assure the flow of transactions', 'Regular audits of the IT system to detect any potential issues', 'Frequent employee training on how to use the IT system', 'Assure that appropriate data backups are stored in an off-site location', 'Frequent changes to the IT system to keep up with technological advancements', 'Ensure that the IT system is updated regularly to avoid any disruptions', 'Review additional expenses to obtain the required amount of business interruption insurance coverage for the organization', 'Maintain appropriate written source documents so the data can be re-entered if it is lost or compromised']\",\"question\":\"In an e-commerce environment that requires that the information technology (IT) system be available on a continuous basis, more emphasis will be placed on which of the following aspects of the planning than in a traditional organization?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}