{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"889715bf-067d-5e72-94ed-ef697bdcdc58","task_key":"default--test--5134","task_revision_id":"1","upstream_id":"5134","short_description":"What was unique about the NATO intervention in Kosovo?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['It was the first time the US became involved in the Balkans', 'It was the first war won by airpower alone', 'It was the first war where NATO suffered no casualties', 'It was the first war in which NATO used drones', 'It was the first time NATO used military force', \\\"It was the first war to employ 'smart weapons'\\\", 'It was the first time NATO was involved in a civil war', 'It was the first time NATO intervened in a non-member state', 'It was the first war where NATO used cyber warfare']\",\"question\":\"What was unique about the NATO intervention in Kosovo?\",\"src\":\"ori_mmlu-us_foreign_policy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}