{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4e2627f-5f18-5147-b2d1-fae962498ef3","task_key":"all--test--b4e2627f-5f18-5147-b2d1-fae962498ef3","task_revision_id":"1","upstream_id":"","short_description":"When coastal States do enforce their criminal jurisdiction over foreign vessels…","config":"all","split":"test","body":"{\"choices\":\"['Port States generally do not enforce their criminal jurisdiction over crimes that do not infringe their customs laws or disrupt peace and public order.', 'Port States enforce their jurisdiction in all cases of delinquent vessels', 'Port States enforce their jurisdiction only in respect of very serious crimes, such as drug trafficking', 'Port States are not entitled to enforce their jurisdiction over any foreign vessel']\",\"question\":\"When coastal States do enforce their criminal jurisdiction over foreign vessels in their ports?\",\"subject\":\"international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}