{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"de79f2e6-1423-5e67-ac80-9f718002dd92","task_key":"default--test--1377","task_revision_id":"1","upstream_id":"1377","short_description":"What dimension did the Kadi judgment introduce with respect to the incorporation…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The Kadi judgment demanded that UNSC resolutions are construed in accordance with human rights', 'The Kadi judgment demanded that all UNSC resolutions be incorporated without any further implementing legislation', 'The Kadi judgment required that important UNSC resolutions be transformed and not merely incorporated', 'The Kadi judgment claimed that UNSC resolutions are not binding if they violate human rights']\",\"question\":\"What dimension did the Kadi judgment introduce with respect to the incorporation of UN Security Council resolutions?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}