{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c844b7ed-c2c3-5a2c-a398-c586a39cd288","task_key":"default--test--6508","task_revision_id":"1","upstream_id":"6508","short_description":"A 35-year-old male presents to the office for the result of an HIV test, and he…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['beneficence is equal to justice', 'autonomy overrides justice', 'autonomy is equal to nonmaleficence', 'autonomy overrides beneficence', 'justice overrides beneficence', 'beneficence overrides justice', 'justice overrides autonomy', 'beneficence overrides autonomy', 'nonmaleficence overrides autonomy', 'nonmaleficence overrides beneficence']\",\"question\":\"A 35-year-old male presents to the office for the result of an HIV test, and he is informed that the result is positive. The patient resists disclosure to his wife because he has not told her about his extramarital sexual partners. The physician informs the patient that his wife's right to know is paramount because she could also be HIV positive. Which of the following relationships best describes the rationale for the physician's recommendation that the patient inform his wife of his condition rather than withhold this information? \",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}