{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6433a10e-ce74-53a7-bf17-67a36f5756c8","task_key":"default--test--10891","task_revision_id":"1","upstream_id":"10891","short_description":"According to Moore, the most fundamental question in all of ethics is:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['how \\\"justice\\\" is to be defined.', 'whether ethics is a science.', 'whether morality is subjective.', 'whether God exists.', 'whether humans are inherently good or bad.', 'how “good” is to be defined.', 'whether life is worth living.', 'what is the meaning of life.', 'whether morality is objective.', 'how \\\"evil\\\" is to be defined.']\",\"question\":\"According to Moore, the most fundamental question in all of ethics is:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}