{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02ba9276-8621-55b0-93da-e8e9c8b70fdd","task_key":"default--test--1055","task_revision_id":"1","upstream_id":"1055","short_description":"Do third States enjoy any fishing rights within the Exclusive Economic Zone of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Third States may never have access to fisheries within another State's EEZ\\\", 'Third States are free to fish within another EEZ, except from certain designated areas', \\\"When the coastal State cannot harvest the 'total allowable catch', the coastal State is to give other State access to that surplus with priority to be given to developing and land-locked States\\\", 'The coastal State must share all of its fish resources with all other States', 'Third States have access to fisheries within its EEZ only during a specific season', 'The coastal State is to give access to fisheries within its EEZ only to States with which it has diplomatic relations', 'Third States have unlimited fishing rights within the EEZ of any State', \\\"Third States can only fish within another State's EEZ with a special permit\\\", 'The coastal State is to give access to fisheries within its EEZ only to neighbouring States', \\\"Third States can only fish within another State's EEZ if they have a mutual agreement\\\"]\",\"question\":\"Do third States enjoy any fishing rights within the Exclusive Economic Zone of another State?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}