{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"320f3553-a2ca-5dd7-b573-86a15e05120d","task_key":"default--test--3220","task_revision_id":"1","upstream_id":"3220","short_description":"Keystone species are thought to have profound effects on the structure and…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['are more abundant than most other species in their communities', 'tend to reduce diversity by eliminating food resources for other species', 'are the largest species in their communities', 'are the most aggressive species in their communities', 'are the only species capable of reproduction', 'have no predators within their communities', 'provide the foundation for food webs', 'are the only species that can adapt to environmental changes', 'can prevent superior competitors from driving inferior competitors to local extinction', 'consume more resources than other species in their communities']\",\"question\":\"Keystone species are thought to have profound effects on the structure and composition of ecological communities because they\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}