{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d87787ff-90ac-5e91-b668-7c7d21d86a36","task_key":"default--test--6308","task_revision_id":"1","upstream_id":"6308","short_description":"A 23-year-old woman comes to the physician for genetic counseling prior to…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The patient is not a carrier of the DMD as her muscle creatine kinase concentration is within the normal range', \\\"The patient's mother is a carrier, hence the patient is definitely a carrier too\\\", 'The patient is a carrier of the disease based on her family history of DMD', \\\"The patient's DMD carrier status is uncertain due to lack of genetic analysis on her deceased relatives.\\\", 'The patient has a 25% risk of having a child with DMD', \\\"The patient is a carrier of the disease based on her mother's elevated creatine kinase concentration\\\", \\\"The patient's DMD carrier status is certain due to maternal transmission\\\", 'The patient has a 50% risk of having a child with DMD', \\\"The patient's DMD carrier status is uncertain because of random X inactivation\\\", 'The patient is not a carrier of the DMD based on her normal creatine kinase concentration']\",\"question\":\"A 23-year-old woman comes to the physician for genetic counseling prior to conception. Her brother and maternal uncle had Duchenne muscular dystrophy (DMD) and died at the ages of 28 and 17 years, respectively. Genetic analysis was not performed on either relative prior to death. Serum studies show a muscle creatine kinase concentration of 120 U/L (N=22– 198). The patient's 50-year-old mother has a serum muscle creatine kinase concentration of 300 U/L. Which of the following is the most appropriate assessment of this patient's carrier status for this disease?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}