{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"052db120-06a5-5587-944f-9f6c7621c500","task_key":"default--test--11235","task_revision_id":"1","upstream_id":"11235","short_description":"Locke can be thought of as accepting a","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['theory of existentialism.', 'version of utilitarianism.', 'theory of subjectivism.', 'Kantian theory.', 'version of virtue ethics.', 'version of consequentialism.', 'Hegelian philosophy.', 'Nietzschean perspective.', 'concept of moral relativism.', 'natural law conception of morality.']\",\"question\":\" Locke can be thought of as accepting a\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}