{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"361e2dde-3a76-5bff-833b-33fb678b3627","task_key":"default--test--2967","task_revision_id":"1","upstream_id":"2967","short_description":"What unusual feature of the cycads makes them are remarkable group among the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They have flowers that bloom for several years before producing seeds', 'They have a unique leaf structure', 'They can survive in extremely cold climates', 'They produce large seeds', 'Their seeds can remain dormant for up to a century before germinating', 'They possess the ability to move their leaves towards the sun', 'They form motile, swimming sperm within their pollen tubes', 'They can photosynthesize at night', 'They exhibit true secondary growth similar to woody trees', 'They are the only seed plants capable of asexual reproduction']\",\"question\":\"What unusual feature of the cycads makes them are remarkable group among the seed plants?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}