{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"560f5945-c69e-5a82-aa07-ebc12b3856b2","task_key":"default--test--7105","task_revision_id":"1","upstream_id":"7105","short_description":"As a result of automatic stabilizers during economic expansions government…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['remain constant and taxes rise.', 'fluctuate and taxes rise.', 'fall and taxes remain constant.', 'rise and taxes fall.', 'and taxes rise.', 'and taxes fall.', 'fall and taxes rise.', 'fall and taxes fluctuate.', 'rise and taxes remain constant.', 'remain constant and taxes fall.']\",\"question\":\"As a result of automatic stabilizers during economic expansions government expenditures\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}