{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8578d8f3-7155-593d-a06b-3820d8954292","task_key":"default--test--3488","task_revision_id":"1","upstream_id":"3488","short_description":"Which of the following pieces of evidence most strongly supports the common…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['All organisms reproduce.', 'All organisms are capable of evolving.', 'All organisms require energy.', 'All organisms have DNA.', 'All organisms are composed of cells.', 'All organisms respire.', 'All organisms use essentially the same genetic code.', 'All organisms show heritable variation.', 'All organisms can move.', 'All organisms require water to survive.']\",\"question\":\"Which of the following pieces of evidence most strongly supports the common origin of all life on Earth?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}