{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4351a65f-7719-5077-bf81-f00adb70b011","task_key":"default--test--11210","task_revision_id":"1","upstream_id":"11210","short_description":"Baier argues that genuine moral rules:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['must take into account the interests of all living beings.', 'make take into account the interests of all sentient beings.', 'should primarily focus on preserving the natural environment.', 'must align with societal norms and expectations.', 'are based solely on religious doctrines.', 'must be for the good of human beings.', 'should be universally applicable regardless of cultural differences.', 'must only consider the interests of the individual making the moral decision.', 'are primarily directed toward promoting self-interest.', 'are only applicable to those who choose to follow them.']\",\"question\":\"Baier argues that genuine moral rules:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}