{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4772599b-fc7d-5a36-9427-a7f3e41d575e","task_key":"default--test--6108","task_revision_id":"1","upstream_id":"6108","short_description":"A 30-year-old woman, gravida 2, para 0, aborta 1, at 28 weeks' gestation comes…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Development of polycythemia', 'Formation of antibodies to RhB', 'Development of lymphocytosis', 'Formation of antibodies to RhO', 'immune globulin to the patient. This treatment is most likely to prevent which of the following in this mother?', 'Development of leukocytosis', 'Formation of antibodies to RhD', 'Development of anemia', 'Formation of antibodies to RhC', 'Development of natural killer cells']\",\"question\":\"A 30-year-old woman, gravida 2, para 0, aborta 1, at 28 weeks' gestation comes to the office for a prenatal visit. She has had one previous pregnancy resulting in a spontaneous abortion at 12 weeks' gestation. Today, her vital signs are within normal limits. Physical examination shows a uterus consistent in size with a 28-week gestation. Fetal ultrasonography shows a male fetus with no abnormalities. Her blood group is O, Rh-negative. The father's blood group is B, Rh-positive. The physician recommends administration of Rho\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}