{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dec8a5b1-e530-5619-9f28-2ca4f38fc6d0","task_key":"default--test--7490","task_revision_id":"1","upstream_id":"7490","short_description":"A balance of trade surplus can be the result of","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['low levels of income relative to other nations.', \\\"a decrease in the country's production.\\\", \\\"foreigners having no taste for this country's products.\\\", 'a loose monetary policy.', 'high levels of income relative to other nations.', \\\"an appreciation of the country's currency.\\\"]\",\"question\":\"A balance of trade surplus can be the result of\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}