{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7267ea1-d7d2-536e-9b5a-90e2db972909","task_key":"default--test--6015","task_revision_id":"1","upstream_id":"6015","short_description":"The patient's self-report of pain is important because health care staff:","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"[\\\"can determine the patient's pain threshold.\\\", \\\"can overestimate the intensity of the patient's pain.\\\", \\\"can compare it with other patients' pain levels.\\\", 'do not possess an objective way of assessing pain.', 'can use the information to prescribe and administer the maximum possible amount of pain relief.', \\\"underestimate the intensity of the patient's pain.\\\", \\\"are able to visually assess the patient's pain level.\\\", \\\"can measure the patient's pain with medical instruments.\\\", 'can use the information to prescribe and administer the least possible amount of pain relief.', \\\"can ignore the patient's pain.\\\"]\",\"question\":\"The patient's self-report of pain is important because health care staff:\",\"src\":\"ori_mmlu-clinical_knowledge\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}