{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47a1ce51-6620-55f9-8314-a864e01690df","task_key":"default--test--10831","task_revision_id":"1","upstream_id":"10831","short_description":"Which of the following claims would Leopold clearly accept?","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['The problems associated with how we should use our natural environment ultimately concern how human beings should treat each other.', 'We should think in terms of \\\"the balance of nature\\\" to properly appreciate the value of the natural environment.', 'The problems associated with how we should use our natural environment are not merely economic problems.', 'We should ignore all economic motives in deciding questions about land use.']\",\"question\":\" Which of the following claims would Leopold clearly accept?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}