{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aeb777fd-c4c7-5853-8672-171d08bc2b17","task_key":"all--test--aeb777fd-c4c7-5853-8672-171d08bc2b17","task_revision_id":"1","upstream_id":"","short_description":"With which of the following statements would both Thomas Hobbes and John Locke…","config":"all","split":"test","body":"{\"choices\":\"['Government authority is created as the result of a social contract.', 'Government must enforce religious law to prevent moral decay.', \\\"Government must enforce the majority's will regardless of the wishes of the minority.\\\", 'Government must bend to the will of the educated minority.']\",\"question\":\"With which of the following statements would both Thomas Hobbes and John Locke most probably have agreed?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}