{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22dd401c-b6c4-5f20-85ce-b5f58da8ac81","task_key":"default--test--11069","task_revision_id":"1","upstream_id":"11069","short_description":"Anscombe claims that an adequate moral psychology would include:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['a detailed understanding of societal norms and expectations.', 'a comprehensive review of historical philosophical theories.', 'the influence of personal experiences on moral decisions.', 'analyses of concepts such as “action” and “intention.”', 'the integration of religious beliefs into moral decisions.', 'a thorough study of the subconscious mind.', 'all of the above.', \\\"an exploration of the concept of 'good' and 'evil'.\\\", 'an answer to the mind-body problem.', 'an explanation of how free will is possible.']\",\"question\":\"Anscombe claims that an adequate moral psychology would include:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}