{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e5251e0-7dc4-5bc8-a5f9-bd5701160856","task_key":"default--test--2961","task_revision_id":"1","upstream_id":"2961","short_description":"The teeth of different species of vertebrates are special-ized in a variety of…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Snakes have thin, sharp teeth for capturing prey; carnivorous mammals like cats and dogs have pointed teeth for cutting and shearing; herbivorous animals like cows and horses have large, flat teeth for breaking up plant cell walls.', 'Fish have teeth adapted for chewing and grinding similar to humans', 'Only carnivorous animals have specialized teeth', 'Snakes have large, flat teeth for grinding food', 'Birds have well-developed teeth for tearing meat', 'Insectivorous mammals have large, flat teeth for grinding plants', 'Rodents have fangs for injecting venom into prey', 'All vertebrates have the same number of teeth', 'Amphibians have teeth specialized for filter-feeding like some whales', 'All vertebrates have similarly structured teeth']\",\"question\":\"The teeth of different species of vertebrates are special-ized in a variety of ways and may be quite unlike those of man in number , structure, arrangement, and function. Give examples .\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}