{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc6ca9cc-9dfc-5a00-8b1d-2b7c4cb89413","task_key":"default--test--11234","task_revision_id":"1","upstream_id":"11234","short_description":"In Baxter's view, the first and foremost step toward a solution of our…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the introduction of new species to balance the ecosystem.', 'maintaining current levels of pollution.', 'an overall maximally clean environment.', 'the total elimination of harmful species.', 'some maximal state of pollution.', 'the eradication of all pollution.', 'an environment free from human interference.', 'a state of pollution that supports human health and well-being.', 'some optimal state of pollution.', 'the preservation of all living species.']\",\"question\":\" In Baxter's view, the first and foremost step toward a solution of our environmental problems is a clear recognition that our objective is not pure air or water but rather\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}