{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e9aa9bb-5f0d-58a4-895e-977da43f9d29","task_key":"default--test--11111","task_revision_id":"1","upstream_id":"11111","short_description":"Stevenson claims that the primary use of ethical judgments is to:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['demonstrate superior knowledge.', 'provide a basis for legislation.', 'evaluate the morality of others.', 'influence the interests of others.', 'state facts.', 'establish universal moral laws.', 'describe one’s own approval of things.', 'express personal beliefs.', 'none of the above.', 'manipulate the emotions of others.']\",\"question\":\"Stevenson claims that the primary use of ethical judgments is to:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}