{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a546911-b8b4-5ef5-8dcb-1e96a3044084","task_key":"default--test--5038","task_revision_id":"1","upstream_id":"5038","short_description":"How long ago did human groups begin actively controlling their food sources by…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['within the past 12,000 years', 'within the past 1,000 years', 'within the past 24,000 years', 'within the past 2,000 years', 'within the past 6,000 years', 'within the past 40,000 years', 'within the past 28,000 years', 'within the past 30,000 years', 'within the past 35,000 years', 'within the past 18,000 years']\",\"question\":\"How long ago did human groups begin actively controlling their food sources by artificially producing conditions under which these sources would grow?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}