{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6eb95e57-b7c0-59e0-839f-e635ecf7ede8","task_key":"default--test--2580","task_revision_id":"1","upstream_id":"2580","short_description":"Compared to individuals in their 20s, which of the following is an advantage to…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Older mothers have fewer medical complications during pregnancy and birth', 'Older parents have more physical energy and time to play with their young children', 'Older parents typically have more income for additional expenses due to established careers', 'Older parents have not had time to build up expectations about children’s behavior', 'Older parents are more likely to live longer and see their children grow up', 'Older parents are less likely to experience postpartum depression', 'Older parents are less likely to have other children to take care of', 'Older parents typically have larger homes to accommodate children']\",\"question\":\"Compared to individuals in their 20s, which of the following is an advantage to having children after the age of 30?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}