{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"645d730c-c3b6-5872-864b-5f5a9a4fda12","task_key":"default--test--1761","task_revision_id":"1","upstream_id":"1761","short_description":"What is the 'Lotus principle'?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The so-called Lotus principle is that customary law is superior to treaty law', 'The so-called Lotus principle is that States have the right to intervene in the internal affairs of other States if it concerns their national security', 'The so-called Lotus principle is that States are permitted to apply their own domestic laws in disputes with other States', 'The so-called Lotus principle is that international law governs the relations between States and not the relations between individuals.', 'The so-called Lotus principle is that States have unlimited jurisdiction over their own citizens, regardless of where they are located', \\\"The so-called Lotus principle is that States are free to ignore international law as long as they don't infringe on the sovereignty of other States\\\", \\\"The so-called Lotus principle is that 'restrictions upon the independence of States cannot therefore be presumed', or, as it has been construed, 'whatever is not prohibited is permitted in international law'\\\", 'The so-called Lotus principle is that States are not prohibited to assert their enforcement jurisdiction on the high seas', 'The so-called Lotus principle is that customary law derives from the combination of State practice and opinio juris', 'The so-called Lotus principle is that States are free to choose the Court that they will submit their disputes']\",\"question\":\"What is the 'Lotus principle'?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}