{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c7a8a3b-7b3f-5462-96f3-46537e810c91","task_key":"default--test--6281","task_revision_id":"1","upstream_id":"6281","short_description":"Which of the following is true of the spine?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"[\\\"Schober's test is designed to quantify the flexion of the lumbar spine\\\", 'The lumbar spine is the upper part of the spine', 'The spine is incapable of healing after injury', 'The human spine has 20 vertebrae', 'Pregnancy is a cause of loss of lordosis of the lumbar spine', 'The spine has no role in movement and balance', 'Ankylosing spondylitis is a cause of hypermobility of the spine', 'Scoliosis is a condition where the spine curves to the left', 'Back pain is an uncommon complaint', 'The cervical spine is the thickest part of the spine']\",\"question\":\"Which of the following is true of the spine?\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}