{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d780a86-4cdd-58e7-b3d0-45a2ae2751c8","task_key":"default--test--6039","task_revision_id":"1","upstream_id":"6039","short_description":"A 78-year-old male presents to the office with the complaint of severe neck…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['the neck was treated in simultaneous hyperrotation and extension', 'the treatment was performed without first conducting a thorough physical examination', 'the treatment was performed without prior consent from the patient', \\\"the physician failed to recognize the patient's age as a risk factor for complications\\\", 'the patient was previously treated for osteoarthritis of the cervical spine', 'the patient was not properly instructed on aftercare following the treatment', 'the physician did not ask about any pre-existing conditions before treatment', 'the patient was not advised of potential risks before the treatment', 'the motion barrier was articulated more than 3 times', 'the motion barrier was met with a vectored force from the direction of ease']\",\"question\":\"A 78-year-old male presents to the office with the complaint of severe neck pain. He rates the pain as a 7 on a scale of 0 to 10 scale. He denies any trauma but reports that he had been doing a lot of painting over the past week. He denies any radicular pain or numbness. Cervical range of motion is moderately limited in all fields. He is treated with articulatory springing technique. Four hours later the physician receives a call from the local emergency department, where the patient presented confused, nauseated, and vertiginous. The physician may be found liable if\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}