{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"265fe448-f226-5fa3-a6e6-73c8cb9aae12","task_key":"default--test--3434","task_revision_id":"1","upstream_id":"3434","short_description":"Unlike large populations, small populations are vulnerable to various processes…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Inbreeding and loss of genetic variation threaten a population.', 'Increased competition for resources within the population threatens a population.', 'Overpopulation is a direct threat to the survival of a population.', 'Weather and climate changes have no impact on small populations.', 'Mutation reduces the health of a population.', 'Small populations are not affected by diseases.', 'The growth of human settlements and industries does not impact small populations.', 'Migration of new individuals into the population threatens a population.', 'Breeding with individuals from a different population may cause the extinction of the first population due to a decrease in diversity.']\",\"question\":\"Unlike large populations, small populations are vulnerable to various processes that draw populations down an extinction vortex toward smaller and smaller populations until no individuals survive. Which of the following statements correctly identifies the factors that endanger a population?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}