{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"664903b7-786a-5b48-9ccb-b7cd28163079","task_key":"default--test--11279","task_revision_id":"1","upstream_id":"11279","short_description":"Concerning the relationship between morality and theology, Bentham claims that:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['God exists, but does not concern himself with matters of morality.', \\\"Morality is always aligned with God's will.\\\", \\\"God's will can be determined independently of morality.\\\", \\\"The concept of right and wrong is solely determined by God's will.\\\", 'we must first know whether something is right before we can know whether it conforms to God’s will.', 'we must first know whether something conforms to God’s will before we can know that it is right.', 'There is no correlation between morality and theology.', 'God does not exist.', \\\"Morality exists independently of God's will.\\\", \\\"God's will is not related to matters of morality.\\\"]\",\"question\":\"Concerning the relationship between morality and theology, Bentham claims that:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}