{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd594f89-a3df-5aa5-a5fa-0a1123e99958","task_key":"default--test--3014","task_revision_id":"1","upstream_id":"3014","short_description":"Although it is possible to categorize eye color in humans into blue or brown…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Polygenic inheritance without any modifiers', 'Genetic drift', 'Mutation only', 'Codominance of two genes', 'Multiple genes with a single modifier', 'Two principal genes', 'One principal gene influenced by modifier genes', 'A single gene with complete dominance', 'Epistatic interactions between unrelated genes', 'Environmental factors only']\",\"question\":\"Although it is possible to categorize eye color in humans into blue or brown phenotypic classes, there are clearly many variations in hue within the two classes. What type of inheritance mechanism may be working to produce eye color?\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}