{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"786e984e-a0a9-548f-a78c-e9bf0af9d598","task_key":"default--test--11090","task_revision_id":"1","upstream_id":"11090","short_description":"In Carruther's view, the justification for moral criticism for things that we do…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"the animal's lifespan.\\\", \\\"the animal's direct moral standing.\\\", 'societal norms and expectations.', \\\"the animal's role in the ecosystem.\\\", \\\"the animal's inability to defend itself.\\\", 'the bad qualities of character that our actions evince.', \\\"the animal's intelligence level.\\\", \\\"the animal's capacity for emotion.\\\"]\",\"question\":\" In Carruther's view, the justification for moral criticism for things that we do (or don't to do) to an animal derives from\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}