{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd26a1ef-aee9-5d39-ac59-5e0319193676","task_key":"default--test--1296","task_revision_id":"1","upstream_id":"1296","short_description":"Which one of the statements below is false?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The war crimes trials at Nuremberg and Tokyo took place immediately after the First World War', 'The war crimes trials found that \\\"superior orders\\\" could not be used as a defence for war crimes or crimes against humanity', \\\"The war crimes trials introduced a 'crime against humanity' as an offence\\\", 'The war crimes trials were held after the Second World War, but before the creation of the United Nations', 'The war crimes trials were conducted by the United Nations', 'The war crimes trials at Nuremberg and Tokyo took place in the United States', 'The war crimes trials did not include the crime of genocide', 'The war crimes trials held individuals legally responsible for acts such as war crimes and crimes against humanity']\",\"question\":\"Which one of the statements below is false?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}