{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfac19b5-ab51-52cc-a406-8c3644abd1b7","task_key":"default--test--10829","task_revision_id":"1","upstream_id":"10829","short_description":"One way to value a patient's autonomy is by aiming to maximize his or her…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"the patient's ability to make informed decisions\\\", 'the ability to share reasons with the patient', \\\"the ability to understand the patient's needs\\\", \\\"the patient's right to refuse treatment\\\", 'the ability to provide the patient with options', \\\"the patient's personal preferences\\\", \\\"the patient's capacity for self-determination\\\", \\\"the patient's opportunities for self-determination\\\", \\\"the patient's understanding of their medical condition\\\", 'the ability to communicate effectively with the patient']\",\"question\":\" One way to value a patient's autonomy is by aiming to maximize his or her effective options. When we value a patient's autonomy in this way, what, according to Velleman, do we value?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}