{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbddb434-ff8a-5c49-b986-4a8b9cd470f4","task_key":"default--test--6106","task_revision_id":"1","upstream_id":"6106","short_description":"A 35-year-old man comes to the office because of 1-week history of mid low back…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Piriformis syndrome', 'Displacement of the nucleus pulposus', 'Compression fracture of the spine', 'Ankylosing spondylitis', 'Osteophyte formation', 'Spondylolisthesis', 'Lumbar spinal stenosis', 'Sacroiliac joint dysfunction', 'Hypertrophy of the facet joints']\",\"question\":\"A 35-year-old man comes to the office because of 1-week history of mid low back pain that radiates down his right leg. The pain began after the patient lifted a heavy box onto his truck. He rates his current pain as an 8 on a 10-point scale. He has been unable to find a comfortable position and has been sleeping in a recliner. Medical history is unremarkable and he takes no medications. He has smoked one pack of cigarettes daily for the past 25 years, and he drinks a six-pack of beer on Friday and Saturday nights. BMI is 27 kg/m2 . He appears uncomfortable and stands during the physical examination. Vital signs are normal. Straight-leg raise test is positive on the right, with loss of right ankle reflex. The remainder of the physical examination discloses no abnormalities. Which of the following is the most likely explanation for this patient’s symptoms?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}