{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"487dc16e-3e96-5a4a-bab1-d735efceb3c0","task_key":"default--test--6323","task_revision_id":"1","upstream_id":"6323","short_description":"Diagnosis of iron deficiency can be complicated by concurrent infection since…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Low haemoglobin, low ferritin, normal serum transferrin receptors, high hepcidin', 'Low haemoglobin, high ferritin, low serum transferrin receptors, low hepcidin', 'Normal haemoglobin, high ferritin, high serum transferrin receptors, low hepcidin', 'Low haemoglobin, high ferritin, normal serum transferrin receptors, low hepcidin', 'Low haemoglobin, low ferritin, high serum transferrin receptors, low hepcidin', 'High haemoglobin, high ferritin, high serum transferrin receptors, high hepcidin', 'Low haemoglobin, high ferritin, high serum transferrin receptors, high hepcidin', 'Low haemoglobin, low ferritin, low serum transferrin receptors, high hepcidin', 'High haemoglobin, low ferritin, low serum transferrin receptors, high hepcidin', 'Normal haemoglobin, low ferritin, high serum transferrin receptors, high hepcidin']\",\"question\":\"Diagnosis of iron deficiency can be complicated by concurrent infection since many markers of iron status are altered by infection. Which of the following combinations of iron status markers is likely to be found in a person with both iron deficiency and a severe infection?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}