{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ed89679-2408-5f17-ad8f-7d0c77de6b51","task_key":"all--test--9ed89679-2408-5f17-ad8f-7d0c77de6b51","task_revision_id":"1","upstream_id":"","short_description":"Hart writes: '[T]he certification of something as legally valid is not…","config":"all","split":"test","body":"{\"choices\":\"['That legal positivism is unconcerned about the morality of the law.', 'That validly enacted law should always be obeyed.', 'That moral issues fall outside the official legal system.', 'That there is no moral duty to obey an unjust law.']\",\"question\":\"Hart writes: '[T]he certification of something as legally valid is not conclusive of the question of obedience ... however great the aura of majesty or authority which the official system may have, its demands must in the end be submitted to a moral scrutiny.' What does this say about the nature of legal positivism?\",\"subject\":\"jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}