{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99e788f8-5f4b-5993-9636-71449f5d27c1","task_key":"default--test--10906","task_revision_id":"1","upstream_id":"10906","short_description":"Baxter argues that we should view our treatment of the environment as a matter…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['to minimize damage incurred by all individual living beings.', 'to promote the functional integrity of ecosystems.', 'to minimize the suffering of all sentient creatures.', 'to ensure the survival of endangered species.', 'to minimize damage to the ozone layer.', 'to achieve a balance between human needs and wildlife preservation.', 'to promote sustainable development and renewable energy use.', 'to maximize economic benefit.', 'to limit greenhouse gas emissions.', 'to promote human welfare.']\",\"question\":\" Baxter argues that we should view our treatment of the environment as a matter of various trade-offs whose aim is\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}