{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24ffc5b2-e36c-55ed-8009-4c5fcd530c27","task_key":"default--test--6973","task_revision_id":"1","upstream_id":"6973","short_description":"For a given level of government spending the federal government usually…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['surplus expansion surplus expansion', 'deficit recession deficit expansion', 'surplus expansion deficit recession', 'deficit expansion deficit recession', 'surplus expansion surplus recession', 'deficit recession surplus expansion', 'surplus recession deficit expansion', 'surplus recession surplus expansion', 'deficit expansion surplus recession', 'surplus recession deficit recession']\",\"question\":\"For a given level of government spending the federal government usually experiences a budget____during economic____and a budget ______during economic_______.\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}