{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cde47166-1704-5f22-a896-b22e7de32951","task_key":"default--test--1288","task_revision_id":"1","upstream_id":"1288","short_description":"Which statement best describes one of Dworkin's central arguments in Justice for…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The law is the only guide for ethical behaviour.', 'The law dictates what moral values should affect our ethical behaviour.', 'Morality plays no role in the concept of law.', 'Moral arguments operate only in hard cases.', 'Morality only applies to specific, complex cases.', 'Legal principles should not be influenced by moral values.', 'Moral values are both independent and objective.', 'Law and morality are entirely separate entities with no overlap.', 'Morality is subjective and changes over time.', 'Ethical behaviour is independent of moral values.']\",\"question\":\"Which statement best describes one of Dworkin's central arguments in Justice for Hedgehogs?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}