{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1022c5d8-f851-5a37-8014-2192a5a80258","task_key":"default--test--10846","task_revision_id":"1","upstream_id":"10846","short_description":"Hare asserts that ethics and science are similar in that:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['both are infallible and beyond questioning.', 'ethics guides science in its pursuit of knowledge.', 'both are essential components of a worthwhile life.', 'both are completely separate and do not intersect.', 'both seek to provide us with true beliefs about the world.', 'both involve testing general principles by examining their logical consequences.', 'all of the above.', 'both are based solely on personal opinions and beliefs.']\",\"question\":\"Hare asserts that ethics and science are similar in that:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}