{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ddbaa9f-d69a-5581-a9eb-ea89a9112630","task_key":"default--test--6188","task_revision_id":"1","upstream_id":"6188","short_description":"An 18-year-old primigravid woman comes for her initial prenatal visit at 16…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Amniocentesis for measurement of α-fetoprotein concentration', 'Immediate cesarean section', 'Hormone therapy to balance MSAFP levels', 'Monitor patient without taking action', 'Triple screening for MSAFP, serum β-hCG, and serum estriol concentrations', 'Administer a pregnancy test', 'Prescribe oral contraceptives', 'Ultrasonography', 'Repeat measurement of MSAFP concentration', 'Conduct a pelvic exam']\",\"question\":\"An 18-year-old primigravid woman comes for her initial prenatal visit at 16 weeks' gestation. She is not sure about the date of her last menstrual period but says that the pregnancy probably occurred immediately after she stopped taking oral contraceptives 5 months ago. Maternal serum α-fetoprotein (MSAFP) concentration is increased to 3 multiples of the median. Which of the following is the most appropriate next step in management?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}