{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"649d3967-34ea-5579-9819-adba6d4f457d","task_key":"default--test--6109","task_revision_id":"1","upstream_id":"6109","short_description":"A 30-year-old woman, gravida 2, para 0, aborta 1, at 28 weeks' gestation comes…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Formation of antibodies to RhO', 'Formation of antibodies to RhC', 'Development of leukocytosis', 'Development of anemia', 'Development of polycythemia', 'Formation of antibodies to RhD', 'Formation of antibodies to RhB', 'Development of natural killer cells', 'immune globulin to the patient. This treatment is most likely to prevent which of the following in this mother?', 'Development of lymphocytosis']\",\"question\":\"A 30-year-old woman, gravida 2, para 0, aborta 1, at 28 weeks' gestation comes to the office for a prenatal visit. She has had one previous pregnancy resulting in a spontaneous abortion at 12 weeks' gestation. Today, her vital signs are within normal limits. Physical examination shows a uterus consistent in size with a 28-week gestation. Fetal ultrasonography shows a male fetus with no abnormalities. Her blood group is O, Rh-negative. The father's blood group is B, Rh-positive. The physician recommends administration of Rho \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}