{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0470236-92f4-5aa7-aa75-f545b154430a","task_key":"default--test--11130","task_revision_id":"1","upstream_id":"11130","short_description":"In contrast to Garrett Hardin's approach, the Confucian model rejects coercion…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['it views coercion as a violation of human rights', 'it believes in the power of persuasion over coercion', 'the Confucian model believes in voluntary compliance based on mutual respect and understanding.', 'it affirms the autonomy of individuals apart from others.', 'The Confucian model actually accepts coercion.', 'the costs of eliminating a collective action problem are not irrelevant to its solution', 'coercion is seen as a last resort, only to be used in extreme circumstances', 'the Confucian model prioritizes collective action over individual action', 'it cannot lead to real social change, which involves changes of mind as well as action', 'it believes in the intrinsic goodness and perfectibility of human beings']\",\"question\":\" In contrast to Garrett Hardin's approach, the Confucian model rejects coercion because\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}