{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72665a4d-52ed-52d0-8061-5a028550c460","task_key":"default--test--11648","task_revision_id":"1","upstream_id":"11648","short_description":"If a binary code is to represent 110 differentcodewords, calculate the number of…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['8 bits, 100 percent', '9 bits, 88 percent', '4 bits, 80 percent', '5 bits, 75 percent', '8 bits, 85 percent', '6 bits, 60 percent', '5 bits, 50 percent', '7 bits, 97 percent', '7 bits, 89 percent', '6 bits, 85 percent']\",\"question\":\"If a binary code is to represent 110 differentcodewords, calculate the number of bits required and evaluate its efficiency as compared to a decimal system to accomplish the same task.\",\"src\":\"stemez-ElectronicCommunications\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}