{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"503a24ad-25e3-58e5-aef4-7b4fb2491e3e","task_key":"default--test--11094","task_revision_id":"1","upstream_id":"11094","short_description":"Nagel thinks that the core of the absolutist position is that","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['all actions are morally equivalent.', 'it is permissible to harm as a foreseen but unintended consequence of action.', 'the ends justify the means.', 'individuals should always act in their own best interest.', 'the moral worth of an action is determined by its outcome.', 'no action can be morally justified.', 'morality is subjective and varies from person to person.', 'the hostility should be directed at its true object.', 'human persons have a right to life.', 'each person should pursue his or her own happiness.']\",\"question\":\" Nagel thinks that the core of the absolutist position is that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}