{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62b1a69c-af4c-5767-8477-3ef3054840a9","task_key":"default--test--11238","task_revision_id":"1","upstream_id":"11238","short_description":"Callahan cites the Oregon Death with Dignity Legal Defense and the Education…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the findings are suspect because the study was performed by advocates of euthanasia.', 'other studies have found contrary results.', 'the study is biased as it was conducted in Oregon, a state with legalized euthanasia.', \\\"they can't know this to be true without an anonymous survey.\\\", 'even if the finding is true, it is irrelevant to the moral status of euthanasia.', 'the results are irrelevant as they do not consider the psychological impact on the families involved.', 'the findings are inaccurate as the study only covers a short period.', 'the findings are misleading as they do not reflect the global perspective on euthanasia.', 'the study lacks credibility as it was not peer-reviewed.']\",\"question\":\" Callahan cites the Oregon Death with Dignity Legal Defense and the Education Center's finding that \\\"after four full years .. there have been no missteps, abuses or coercive tendencies\\\" regarding laws permitting euthanasia. In response he argues that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}