{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f047d4e0-29f4-58db-9c12-af3cdab4b4a7","task_key":"default--test--10865","task_revision_id":"1","upstream_id":"10865","short_description":"Hourdequin argues that one flaw of consequentialist calculation is that","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['it negates the notion that individuals are inherently self-interested actors.', 'it can run counter to the idea that persons are atomistic and individually rational actors.', 'it overlooks the potential conflict between individual and collective interests.', \\\"it can run counter to a person's being able to integrate her commitments at various levels.\\\", 'it fails to reconcile prima facie duties.', \\\"it doesn't consider the potential of individuals to act against their own self-interest.\\\", 'it undermines the importance of personal values and beliefs in decision-making.', 'it disregards the impact of social norms on individual decision-making.', 'it fails to recognize that possibility that the persons are best understood in relation to one another.', 'it discredits the notion that individuals can make morally autonomous decisions.']\",\"question\":\" Hourdequin argues that one flaw of consequentialist calculation is that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}