{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9141c3c2-c2e5-53d9-8707-814fa50cd388","task_key":"default--test--10866","task_revision_id":"1","upstream_id":"10866","short_description":"Biocentrism is the view that","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the primary bearers of direct moral standing are ecosystems in virtue of their functional integrity.', 'the only beings that possess direct moral standing are human beings.', 'the primary bearers of direct moral standing are individual species.', 'only beings that can feel pain have direct moral standing.', 'all beings, because they exist, possess direct moral standing.', 'only beings that possess consciousness have direct moral standing.', 'only human beings and animals have direct moral standing.', 'all and only sentient creatures have direct moral standing.', 'all living beings, because they are living, possess direct moral standing.', 'all beings, living or non-living, possess direct moral standing.']\",\"question\":\" Biocentrism is the view that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}