{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f77e125-2475-5bcb-8af4-ddb6befb99ec","task_key":"default--test--10651","task_revision_id":"1","upstream_id":"10651","short_description":"Consider the following assumptions about electronic storage for the text in all…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Kilobyte (2^10 or approximately 10^3 bytes)', 'Petabyte ( 2^50 or approximately 10^15 bytes)', 'Yottabyte (2^80 or approximately 10^24 bytes)', 'Exabyte (2^60 or approximately 10^18 bytes)', 'Byte (1 byte)', 'Bit (1/8 of a byte)', 'Zettabyte (2^70 or approximately 10^21 bytes)', 'Terabyte ( 2^40 or approximately 10^12 bytes)', 'Gigabyte ( 2^30 or approximately 10^9 bytes)', 'Megabyte ( 2^20 or approximately 10^6 bytes)']\",\"question\":\"Consider the following assumptions about electronic storage for the text in all the books of a university's libraries. - The libraries at the university collectively contain 3 million books. - A book contains an average of 400 pages. - A page contains an average of 50 lines. - A line on a page contains an average of 10 words. - A word contains an average of 5 letters/characters. - A letter/character is represented by 1 byte. Based on the given assumptions, which of the following is the unit in which the electronic storage required for the text in all the books of the university's libraries would best be measured?\",\"src\":\"ori_mmlu-high_school_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}