{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77958bdf-ffbb-5a50-bb7d-4e0330ee8deb","task_key":"default--test--11089","task_revision_id":"1","upstream_id":"11089","short_description":"Which of the following statements best describes Gill's response to the argument…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['Doctors are normally required to make life-and-death decisions, and this is no exception.', \\\"The law is in line with the doctors' Hippocratic oath of doing no harm.\\\", 'Doctors are as qualified as any other mature adult to decide whether a life is worth living.', \\\"The law doesn't require doctors to judge the worth of a life, but rather to provide a professional opinion on the patient's condition.\\\", 'The argument is valid because doctors should not be the ones to decide whether a life is worth living.', 'The law is justified as doctors are inherently equipped to make such judgements due to their profession.', \\\"The law only requires doctors to make medical decisions about the patient's life expectancy and prognosis.\\\", 'The argument is self-defeating because it was proposed by medical doctors.', \\\"The argument is irrelevant as the law is about patient autonomy and not about doctors' judgement on life's worth.\\\", 'The law is flawed because it puts too much pressure on the doctors to make such decisions.']\",\"question\":\" Which of the following statements best describes Gill's response to the argument that the Oregon physician-assisted law falsely assumes that doctors are qualified to judge whether a life is worth living?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}