{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fc96092-8092-5615-b228-31989a980249","task_key":"default--test--6050","task_revision_id":"1","upstream_id":"6050","short_description":"Which of the following is a function of biotin?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Oxidation of glucose to carbon dioxide and water', 'Conversion of pyruvate to lactate', 'Synthesis of methionine from homocysteine', 'Synthesis of purine nucleotides', 'Decarboxylation of amino acids to form amine neurotransmitters', 'Synthesis of fatty acids from acetyl-CoA', 'Conversion of phenylalanine to tyrosine', 'Conversion of glucose to glycogen', 'Carboxylation of pyruvate to oxaloacetate', 'Synthesis of methylene tetrahydrofolate']\",\"question\":\"Which of the following is a function of biotin?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}