{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14e9d72f-877b-5a6e-b364-922d1feaa84e","task_key":"default--test--11019","task_revision_id":"1","upstream_id":"11019","short_description":"Wolf claims that interpreting Kant’s ethics so that it consists of a finite set…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['contradicts the basic principles of Kantian ethics.', 'simplifies the complexity of Kantian ethics.', 'transforms the theory into a form of contractarianism.', 'makes it too lenient and flexible.', 'yields an unattractive picture of moral sainthood.', 'distorts the fundamental nature of the theory.', 'makes the theory too abstract and impractical.', 'implausibly places an “upper bound” on moral worthiness.', 'yields a theory that is too demanding.', 'results in a philosophical dilemma.']\",\"question\":\"Wolf claims that interpreting Kant’s ethics so that it consists of a finite set of constraints:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}