{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"549e90e8-8be1-5ba9-8f6e-cf1cf515e770","task_key":"all--test--549e90e8-8be1-5ba9-8f6e-cf1cf515e770","task_revision_id":"1","upstream_id":"","short_description":"In what ways is the case for humanitarian intervention, as outlined in this…","config":"all","split":"test","body":"{\"choices\":\"['It is not self-evident that individuals do have universal and fundamental human rights.', 'Advocates of humanitarian intervention exaggerate the extent of global consensus about the use of force to protect human rights.', 'Any norm endorsing the use of force to protect individual rights could be abused by powerful states.', 'All of these options.']\",\"question\":\"In what ways is the case for humanitarian intervention, as outlined in this chapter, limited?\",\"subject\":\"security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}