{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"171138e9-a927-5887-adae-1ed980b1f70a","task_key":"default--test--3472","task_revision_id":"1","upstream_id":"3472","short_description":"A newborn infant is examined by a doctor, who determines that the child has…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['An infected mother can transmit syphilis by placental transfer to the fetus during the first four months of pregnancy or as the fetus passes through the infected vagina during birth.', 'The infant contracted syphilis through exposure to infected water during the first bath', 'The infant contracted syphilis from a contaminated vaccine', 'The infant contracted syphilis from sharing toys with other infected children', 'The infant contracted syphilis from an infected nurse', 'The infant contracted syphilis from an unsanitary hospital environment', 'The infant contracted syphilis from a genetic predisposition inherited from the parents', 'The infant contracted syphilis from a blood transfusion shortly after birth', 'The infant contracted syphilis from contact with an infected pet at home']\",\"question\":\"A newborn infant is examined by a doctor, who determines that the child has syphilis. Explain how the child con-tracted syphilis .\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}