{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c56ff448-8a0b-5137-a8b7-2de0a8d5e0b0","task_key":"default--test--6306","task_revision_id":"1","upstream_id":"6306","short_description":"A 19-year-old male presents to the office for evaluation after he was hit from…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['meniscus tear', 'tibial stress fracture', 'posterior cruciate ligament tear', 'unilateral sacral shear', 'patellar dislocation', 'medial collateral ligament sprain', 'posteriorly deviated fibular head', 'anterior cruciate ligament tear', 'plantar flexed cuboid', 'anteriorly deviated distal femur']\",\"question\":\"A 19-year-old male presents to the office for evaluation after he was hit from behind below the right knee while playing football. Gait analysis reveals a lack of fluid motion. Standing flexion test results are negative. Cruciate and collateral knee ligaments appear intact. Foot drop on the right is noted. The most likely diagnosis is\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}