{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"877fe202-6381-549e-820f-ae4cc2c37fcd","task_key":"default--test--7432","task_revision_id":"1","upstream_id":"7432","short_description":"If the U.S. dollar and Chinese yuan are traded in flexible currency markets…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Rising per capita GDP in China increasing imports from the United States', 'Lower per capita GDP in China decreasing imports from the United States', 'Rising unemployment rates in the United States', 'Higher price levels in the United States relative to China', 'Decrease in the American GDP', 'Lower interest rates in the United States relative to China', 'Growing American preference to consume more Chinese-made goods', 'Lower price levels in China relative to the United States']\",\"question\":\"If the U.S. dollar and Chinese yuan are traded in flexible currency markets which of the following causes an appreciation of the dollar relative to the Chinese yuan?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}