{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"688c1af1-3b10-568b-8b3d-831ea9258679","task_key":"default--test--11492","task_revision_id":"1","upstream_id":"11492","short_description":"A time-division multiplexing system using PCM is used to multiplex telephone…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['45 signals', '90 signals', '80 signals', '50 signals', '150 signals', '100 signals', '30 signals', '75 signals', '120 signals', '60 signals']\",\"question\":\"A time-division multiplexing system using PCM is used to multiplex telephone conversations over a single com-munications channel. It is found that a minimum inter-val of 1\\\\musmust be allowed for reliable identification of bits at the receiving end. If the allowable voice bandwidth for telephone line transmission is 3kHz and the quantization level is given as 16, approximate the number of voice signals that can be multiplexed.\",\"src\":\"stemez-ElectronicCommunications\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}