{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00262271-8bee-54c5-8993-d9222ea542d3","task_key":"default--test--6696","task_revision_id":"1","upstream_id":"6696","short_description":"Which of the following statements about the global ageing population is correct?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['In older age groups men outnumber women.', 'The rate of increase in the median age of populations is slower in developing countries than wealthy countries.', 'The global aging population is decreasing due to advancements in healthcare.', \\\"By 2050, the majority of the world's population will be under the age of 40.\\\", 'Men have a lower mortality rate than women, over the life course.', 'The rate of increase in the median age of populations is greater in developing countries than wealthy countries.', 'In older age groups, the number of men and women is equal.']\",\"question\":\"Which of the following statements about the global ageing population is correct?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}