{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e6d1b89-f64d-5c82-ae41-25a997ae4bef","task_key":"default--test--3324","task_revision_id":"1","upstream_id":"3324","short_description":"Mules are relatively long-lived and hardy organisms that cannot, generally…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They can reproduce asexually.', 'They can produce viable gametes through mitosis.', 'They have a relative evolutionary fitness of zero.', 'Their offspring are always genetically identical to the parents.', 'They cannot produce offspring due to lack of genetic variation.', 'If crossing-over happens in mules, then it must be limited to prophase of mitosis.', 'Their offspring have less genetic variation than the parents.', 'Their DNA is structurally different from other organisms.', 'They have a higher rate of genetic mutation than other organisms.', 'Mutations cannot occur in their genomes.']\",\"question\":\"Mules are relatively long-lived and hardy organisms that cannot, generally speaking, perform successful meiosis. Consequently, which statement about mules is true?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}