{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80826820-63f6-5b3d-afa6-f0ca8f8f0c75","task_key":"all--test--80826820-63f6-5b3d-afa6-f0ca8f8f0c75","task_revision_id":"1","upstream_id":"","short_description":"To claim that one has a prima facie duty to obey the law is to assert that:","config":"all","split":"test","body":"{\"choices\":\"['One is under a duty to obey the law without exception, regardless of its content.', \\\"There is no duty to obey a law to which one hasn't agreed.\\\", 'The duty applies only to those who are directly affected by the law in question.', 'The obligation arises independently of the precise legal provisions involved - unless there are grounds to justify a specific exception.']\",\"question\":\"To claim that one has a prima facie duty to obey the law is to assert that:\",\"subject\":\"jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}