{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8fdfbb11-06a1-57c1-9697-3d7b538ff8fe","task_key":"default--test--1486","task_revision_id":"1","upstream_id":"1486","short_description":"Hume's attack on natural law is founded on his argument that:","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['There is no universal agreement on what constitutes natural law.', 'Natural law is an arbitrary concept.', 'Natural law does not apply to all cultures and societies.', 'Natural law is backward-looking.', 'There is no such thing as a social contract.', 'We cannot objectively know what is morally right or wrong.', 'Natural law is an outdated concept.', 'Natural law is not based on empirical evidence.', 'Natural law fails to protect the state against attacks.', 'Natural law does not give adequate consideration to individual rights.']\",\"question\":\"Hume's attack on natural law is founded on his argument that:\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}