{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0f4996b-73dd-5a44-a9dc-035dc4afd789","task_key":"default--test--3144","task_revision_id":"1","upstream_id":"3144","short_description":"A newborn infant is examined by a doctor, who determines that the child has…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The child contracted it from a medical procedure', 'The child contracted it from another infant', 'The child was infected due to a blood transfusion shortly after birth.', 'The child contracted it through exposure to the bacteria in the hospital air.', 'The child contracted it from an unclean environment', 'The child contracted it from contact with contaminated food or water.', 'The child contracted it through breastfeeding from an infected mother.', 'The child contracted it from a caregiver through non-sexual skin-to-skin contact.', 'The child inherited the bacteria genetically from the parents.', 'An infected mother can transmit the organism by placental transfer to the fetus during the first four months of pregnancy.']\",\"question\":\"A newborn infant is examined by a doctor, who determines that the child has syphilis. Explain how the child con-tracted syphilis .\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}