{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5af9fb6-7221-5929-a441-44c86df16051","task_key":"all--test--d5af9fb6-7221-5929-a441-44c86df16051","task_revision_id":"1","upstream_id":"","short_description":"Is there any hierarchy or priority among the various methods of peaceful…","config":"all","split":"test","body":"{\"choices\":\"['Yes, all the political methods (e.g. negotiation, mediation, inquiry, conciliation) should be exhausted prior to resorting to legal methods', 'Legal methods prevail over the political methods, since they are binding upon the parties', 'There is no hierarchy among these methods and the choice belongs to the disputing States', 'It is a matter of an impartial third party to decide which method will have priority over the other']\",\"question\":\"Is there any hierarchy or priority among the various methods of peaceful settlement of disputes?\",\"subject\":\"international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}