{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d1c4abb-73e4-58f2-ad56-d0fe84139570","task_key":"default--test--1848","task_revision_id":"1","upstream_id":"1848","short_description":"Which is the least accurate description of legal positivism?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['It perceives law as arbitrary and without any logical structure.', 'It regards morals and law as inseparable.', 'It espouses the view that there is no necessary connection between morality and law.', 'It regards a legal order as a closed logical system.', 'It perceives law as commands.', 'It regards law as suggestions rather than commands.', 'It asserts that law is based on personal beliefs and opinions.', 'It implies that legal decisions are made purely on emotional grounds.', 'It views law as inherently just and moral.', 'It suggests that morality is the only basis of law.']\",\"question\":\"Which is the least accurate description of legal positivism?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}