{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a5161a7-1812-526a-a3fa-f162e477e18f","task_key":"default--test--3051","task_revision_id":"1","upstream_id":"3051","short_description":"In most ecosystems, net primary productivity is important because it represents…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['biomass of all producers', 'biomass of all consumers', 'energy available to heterotrophs', 'energy consumed by decomposers', 'total solar energy converted to chemical energy by producers', 'energy available to producers', 'energy available to autotrophs', 'total energy lost during energy transfer between trophic levels', 'total energy produced by autotrophs and heterotrophs combined', 'energy stored in the soil']\",\"question\":\"In most ecosystems, net primary productivity is important because it represents the\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}