{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e13020ee-c956-574d-a962-c3b01aeb308d","task_key":"default--test--7293","task_revision_id":"1","upstream_id":"7293","short_description":"If the demand for dollars rises while the supply of dollars falls then the","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['dollar will remain stable.', 'dollar will lose its status as a global currency.', 'inflation will rise.', 'exchange rates will rise but the value of the dollar will not be affected.', 'value of other currencies will fall.', 'dollar will depreciate.', 'exchange rates will be affected but not the value of the dollar.', 'dollar will appreciate.', 'exchange rate will not be affected.']\",\"question\":\"If the demand for dollars rises while the supply of dollars falls then the\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}