{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a14254e-6cf7-5c3a-97ae-2d771e8297b6","task_key":"default--test--10907","task_revision_id":"1","upstream_id":"10907","short_description":"If it were known that a policy would wipe out several animal species without…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['this policy is morally problematic because it disrupts the natural order.', 'this policy is morally problematic because it negatively impacts biodiversity.', 'this policy is morally acceptable because it does not harm human beings.', 'this policy is morally unproblematic.', 'this policy is morally obligatory because it prioritizes human beings.', 'this policy is morally unacceptable because it eliminates several animal species.', 'this policy is morally obligatory because the environment ought not to exist.', 'this policy is morally problematic because it damages the integrity of the ecosystem.', 'this policy is morally problematic because it harms the environment.']\",\"question\":\" If it were known that a policy would wipe out several animal species without negatively affecting human beings, Baxter would most likely say that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}