{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"117a654e-3f7d-5516-b35a-546fb82a8810","task_key":"default--test--2925","task_revision_id":"1","upstream_id":"2925","short_description":"Which of the following observations helped Darwin shape his concept of descent…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Species diversity declines farther from the equator.', 'Fewer species live on islands than on the nearest continents.', \\\"Birds can be found on islands located farther from the mainland than the birds' maximum nonstop flight distance.\\\", 'South American temperate plants are more similar to the tropical plants of South America than to the temperate plants of Europe.']\",\"question\":\"Which of the following observations helped Darwin shape his concept of descent with modification?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}