{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f58b7517-e41f-55a2-9be2-4585d4ce8928","task_key":"default--test--6507","task_revision_id":"1","upstream_id":"6507","short_description":"A 35-year-old male presents to the office for the result of an HIV test, and he…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['justice overrides autonomy', 'nonmaleficence overrides beneficence', 'beneficence overrides justice', 'autonomy overrides beneficence', 'autonomy is equal to nonmaleficence', 'autonomy overrides justice', 'nonmaleficence overrides autonomy', 'justice overrides beneficence', 'beneficence is equal to justice', 'beneficence overrides autonomy']\",\"question\":\"A 35-year-old male presents to the office for the result of an HIV test, and he is informed that the result is positive. The patient resists disclosure to his wife because he has not told her about his extramarital sexual partners. The physician informs the patient that his wife's right to know is paramount because she could also be HIV positive. Which of the following relationships best describes the rationale for the physician's recommendation that the patient inform his wife of his condition rather than withhold this information?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}