{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbcdb241-0954-5a24-804a-85942b60b7ff","task_key":"default--test--6898","task_revision_id":"1","upstream_id":"6898","short_description":"If interest rates rise relatively more in country A than in country B then the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['depend on the GDP of country A.', \\\"be pegged to the value of country B's currency.\\\", 'appreciate.', 'increase, then decrease.', 'depreciate.', 'fluctuate unpredictably.', 'decrease, then increase.', 'change indeterminately.', 'depend on the inflation rate in country B.', 'remain unchanged.']\",\"question\":\"If interest rates rise relatively more in country A than in country B then the value of country A's currency will\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}