{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c2fbc7a-72e6-552e-bbb9-4ca5d7975d45","task_key":"default--test--2954","task_revision_id":"1","upstream_id":"2954","short_description":"Two races of corn averaging 48 and 72 inches in height are crossed. The height…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Eleven polygenes, 2.2 inches each', 'Three polygenes, 8 inches each', 'Five polygenes, 4.8 inches each', 'Six polygenes, 4 inches each', 'Twelve polygenes, 2 inches each', 'Eight polygenes, 3 inches each', 'Seven polygenes, 3.4 inches each', 'Four polygenes, 6 inches each', 'Nine polygenes, 2.7 inches each', 'Ten polygenes, 2.4 inches each']\",\"question\":\"Two races of corn averaging 48 and 72 inches in height are crossed. The height of the F_1 generation does not vary very much, averaging 60 inches. When 500 of these plants were classified, two were 48 inches, and two were 72 inches, the rest falling between these in height. What is the number of polygenesinvolved, and how many inches does each contribute?\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}