{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5304a04d-aac2-5df8-a3cc-886bc504a8b1","task_key":"default--test--3219","task_revision_id":"1","upstream_id":"3219","short_description":"Which of the following adaptations would limit pollination by bees and promote…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Flowers with a narrow opening that only small insects can access', 'Pendant (hanging) red-colored flowers', 'Modified petals to provide a landing space', 'Flowers with a strong sweet fragrance', 'Bright blue flowers that open in the early morning', 'Small, white, night-blooming flowers', 'Flowers producing large amounts of nectar with low sugar concentration', 'Flowers with spiky petals and deep tubes', 'Patterns of ultraviolet color on the petals', 'Nectar with high sugar concentration produced in limited amounts']\",\"question\":\"Which of the following adaptations would limit pollination by bees and promote hummingbird pollination?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}