{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20ab4d1e-790f-52b8-94fe-ce2db2e5f232","task_key":"default--test--1051","task_revision_id":"1","upstream_id":"1051","short_description":"What was the outcome before the European Court of Human Rights in the Al-Adsani…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The Court held that the right to a fair trial trumped the privilege of immunity', 'The Court held that the privilege of immunity was not applicable in this case', 'The Court held that the right to a fair trial was not applicable in this case', 'The Court held that immunities were not in conflict with the right to a fair trial', 'The Court held that the case was admissible due to overriding human rights considerations', 'The Court held that immunity trumped the right to a fair trial', 'The Court held that the case was inadmissible due to lack of jurisdiction', \\\"The Court held that the case was inadmissible due to the defendant's diplomatic status\\\", 'The Court held that human rights considerations prevailed over all other contrary rules of international law, including most immunities', 'The Court held that both immunity and the right to a fair trial were in conflict with each other']\",\"question\":\"What was the outcome before the European Court of Human Rights in the Al-Adsani case?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}