{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05a0e326-a277-5f68-831a-fe9d4eb4a613","task_key":"default--test--10484","task_revision_id":"1","upstream_id":"10484","short_description":"Which of the following considerations applies (apply) to choosing the page size…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['III only', 'II only', 'Only II and I apply, not III', 'I only', 'I and III only', 'I and II only', 'I, II, and III', 'Only III and I apply, not II', 'II and III only', 'None of the above considerations apply']\",\"question\":\"Which of the following considerations applies (apply) to choosing the page size in a paging system?\\nI. An advantage of larger pages is that they lead to smaller page tables.\\nII. An advantage of smaller pages is that they lead to less waste due to internal fragmentation.\\nIII. Normally, the dominant factor in disk access time is not dependent on page length, so longer pages can be used advantageously.\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}