{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36778f4d-34f3-5cce-9ae2-0c37b60f2f95","task_key":"default--test--6767","task_revision_id":"1","upstream_id":"6767","short_description":"Which of the following is a function of folate?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Synthesis of fatty acids from glucose', 'Conversion of oxaloacetate to phosphoenolpyruvate', 'Carboxylation of pyruvate to oxaloacetate', 'Conversion of fructose to glucose', 'Synthesis of TMP (thymidine monophosphate)', 'Decarboxylation of amino acids to form amine neurotransmitters', 'Synthesis of methionine from homocysteine', 'Oxidation of pyruvate to acetyl CoA', 'Conversion of glucose to pyruvate in glycolysis']\",\"question\":\"Which of the following is a function of folate?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}