{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"310b727c-d94d-56a5-b008-7209e2f518ff","task_key":"all--test--310b727c-d94d-56a5-b008-7209e2f518ff","task_revision_id":"1","upstream_id":"","short_description":"Are the peremptory norms of international law the only norms that set out…","config":"all","split":"test","body":"{\"choices\":\"['Yes', 'No, acts of international organizations also set out such obligations', 'No, decisions of international courts also set out such obligations', 'No, even though all peremptory norms set out obligations erga omnes, the latter may also spring from treaties regulating objective regimes or unilateral acts']\",\"question\":\"Are the peremptory norms of international law the only norms that set out obligations erga omnes?\",\"subject\":\"international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}