{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88934ba0-9b25-53f7-84d5-e5435edb938d","task_key":"all--test--88934ba0-9b25-53f7-84d5-e5435edb938d","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['The Scientific Revolution', 'The Reformation', 'The Enlightenment', 'The Romantic Movement']\",\"question\":\"This question refers to the following information.\\nRead the the following quotation to answer questions.\\nWhat is tolerance? … We are full of weakness and errors; let us mutually pardon our follies. This is the last law of nature. … Of all religions, the Christian ought doubtless to inspire the most tolerance, although hitherto the Christians have been the most intolerant of all men.\\nVoltaire, Letters on the English Nation, 1733\\nFrom the passage, one may infer that Voltaire was participating in what cultural movement?\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}