{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"78dac562-1c9f-5cb7-8550-5f044f39c358","task_key":"default--test--6654","task_revision_id":"1","upstream_id":"6654","short_description":"Which of the following is NOT one of the major reasons for the increase in…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Increases in life expectancy at a specified age', 'The increase in the rate of technological advancements', 'The decrease in childhood mortality rates', 'The steadily increasing birth rate', 'The Baby Boom generation', 'Increases in life expectancy at birth', 'The decline in the youth population']\",\"question\":\"Which of the following is NOT one of the major reasons for the increase in numbers of older adults?\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}