{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"12d9a9fa-39c8-598f-b4c5-a276a2a8d687","task_key":"default--test--3502","task_revision_id":"1","upstream_id":"3502","short_description":"Suppose you were given a cage full of small, rat-like animals from the Rocky…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The two groups are from the same species if they have similar tail lengths.', 'The two groups of animals are from the same species if they react to threats in the same manner.', 'The two groups of animals are from the same species if they eat the same food.', 'The two groups of rat-like animals would be classified as members of the same species if they can breed with one another and produce fertile offspring.', 'The two groups of animals are from the same species if they are from the same geographical region.', 'The two groups are from the same species if they have the same number of chromosomes.', 'The two groups are from the same species if they have similar burrowing habits.', 'The two groups of animals are from the same species if they look alike.', 'The two groups are from the same species if they are active during the same time of day (nocturnal or diurnal).', 'The two groups of animals are from the same species if they can communicate with each other using similar sounds.']\",\"question\":\"Suppose you were given a cage full of small, rat-like animals from the Rocky Mountain region and another cage full of similar animals from the Appalachian Mountains. How would you determine if the two groups were of the same or different species?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}