{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10826a27-313e-55c2-adfd-169c41c7afde","task_key":"default--test--10919","task_revision_id":"1","upstream_id":"10919","short_description":"Instead of asking why the act of destroying the environment might be immoral,…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['how the decision to preserve the environment benefits the environment.', 'how the destruction of the environment affects the economy.', 'why people who preserve the environment might be good people.', 'how destroying the environment affects future generations.', 'why the act of destroying nature might be immoral.', 'whether plants have interests.', 'why people who destroy the environment might be bad people.', 'if destroying the environment is legally punishable.', 'whether nature has a right to exist.', 'whether animals have rights.']\",\"question\":\" Instead of asking why the act of destroying the environment might be immoral, Hill wants to ask\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}