{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1309c50-8cb4-5711-a321-04d754fb42d3","task_key":"default--test--3384","task_revision_id":"1","upstream_id":"3384","short_description":"Which statement about the beak size of finches on the island of Daphne Major…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"[\\\"Each bird's beak size increased due to the abundance of food during the drought.\\\", 'Each bird developed a deeper, stronger beak as the drought persisted.', \\\"Each bird's beak became weaker and narrower as the drought persisted.\\\", 'Each bird evolved a deeper, stronger beak as the drought persisted.', \\\"Each bird's survival was not influenced by the depth and strength of its beak during the drought.\\\", 'Each bird that survived the drought produced only offspring with deeper, stronger beaks than seen in the previous generation.', \\\"Each bird's beak size did not change during the drought.\\\", \\\"Each bird's survival was strongly influenced by the depth and strength of its beak as the drought persisted.\\\", \\\"Each bird's survival was only influenced by its ability to find water during the drought, not the size of its beak.\\\", 'Each bird that survived the drought produced offspring with the same beak size as seen in the previous generation.']\",\"question\":\"Which statement about the beak size of finches on the island of Daphne Major during prolonged drought is true?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}