{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d842827-f2a3-557a-a242-619b4a5b5d5e","task_key":"default--test--11046","task_revision_id":"1","upstream_id":"11046","short_description":"According to the Humanity formulation of Kant's principle, we must treat persons","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['never as a mere means.', 'always as a means to an end.', 'in ways that serve the best interests of the majority.', 'in ways that promote universal happiness.', 'in ways that produce the most good for everyone concerned.', \\\"in ways that serve one's own best interests.\\\", 'never as a means.', 'never as an end in itself.']\",\"question\":\" According to the Humanity formulation of Kant's principle, we must treat persons\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}