{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c704e1f-26a7-5174-91d9-c6e7c8b50aea","task_key":"default--test--10397","task_revision_id":"1","upstream_id":"10397","short_description":"The digest created by a hash function is normally called a","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Message authentication code (MAC)', 'Message digest', 'Message integrity code (MIC)', 'Message alteration detection', 'Data authentication algorithm (DAA)', 'Hash output code (HOC)', 'Modication detection code (MDC)', 'Message authentication cipher', 'Modify authentication connection', 'Message authentication control']\",\"question\":\"The digest created by a hash function is normally called a\",\"src\":\"ori_mmlu-computer_security\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}