{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eec86bec-8014-5aab-854e-5a9547ec1977","task_key":"default--test--11194","task_revision_id":"1","upstream_id":"11194","short_description":"Velleman worries that, by offering a patient the option of euthanasia,","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['we encourage a culture of euthanasia.', 'we violate medical ethics.', \\\"we disregard the patient's will to live.\\\", \\\"we limit the patient's treatment options.\\\", \\\"we harm the patient's family.\\\", 'we harm the patient.', \\\"we infringe on the patient's religious beliefs.\\\", 'we risk legal complications.', 'we allow the patient to harm him- or herself.', 'we force doctors to harm the patient.']\",\"question\":\" Velleman worries that, by offering a patient the option of euthanasia,\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}