{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5ff0113-84e7-5b6b-837e-8647090c7b63","task_key":"default--test--3091","task_revision_id":"1","upstream_id":"3091","short_description":"A newborn infant is examined by a doctor, who determines that the child has…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The baby contracted it from a contaminated surface.', 'The baby acquired the infection through kissing or close contact with an infected person.', 'The infant contracted the disease from another infant.', 'The child inherited the disease as an autosomal recessive trait from the parents.', 'The child contracted the disease from a blood transfusion shortly after birth.', 'The infant was vaccinated with a contaminated needle.', 'An infected mother can transmit the organism by placental transfer to the fetus during the first four months of pregnancy or as the fetus passes through the infected vagina during birth.', 'The infant developed the disease due to exposure to the bacterium in the hospital environment after birth.', 'The baby contracted it through breastfeeding from an infected mother.', 'The baby was born with it due to a genetic mutation.']\",\"question\":\"A newborn infant is examined by a doctor, who determines that the child has syphilis. Explain how the child con-tracted syphilis .\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}