{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51afea85-2a30-52ae-93fc-599ea1801251","task_key":"default--test--7104","task_revision_id":"1","upstream_id":"7104","short_description":"Suppose that households increase the demand for U.S. Treasury bonds as financial…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decreased supply \\\\xa0\\\\xa0\\\\xa0 Rising \\\\xa0\\\\xa0\\\\xa0 Depreciates', 'Decreased supply \\\\xa0\\\\xa0\\\\xa0 Falling \\\\xa0\\\\xa0\\\\xa0 Depreciates', 'Increased supply \\\\xa0\\\\xa0\\\\xa0 Rising \\\\xa0\\\\xa0\\\\xa0 Depreciates', 'Decreased demand \\\\xa0\\\\xa0\\\\xa0 Falling \\\\xa0\\\\xa0\\\\xa0 Appreciates', 'Increased supply \\\\xa0\\\\xa0\\\\xa0 Falling \\\\xa0\\\\xa0\\\\xa0 Appreciates', 'Decreased supply \\\\xa0\\\\xa0\\\\xa0 Rising \\\\xa0\\\\xa0\\\\xa0 Appreciates', 'Increased demand \\\\xa0\\\\xa0\\\\xa0 Rising \\\\xa0\\\\xa0\\\\xa0 Appreciates', 'Increased supply \\\\xa0\\\\xa0\\\\xa0 Rising \\\\xa0\\\\xa0\\\\xa0 Appreciates', 'Increased demand \\\\xa0\\\\xa0\\\\xa0 Falling \\\\xa0\\\\xa0\\\\xa0 Depreciates', 'Decreased demand \\\\xa0\\\\xa0\\\\xa0 Falling \\\\xa0\\\\xa0\\\\xa0 Depreciates']\",\"question\":\"Suppose that households increase the demand for U.S. Treasury bonds as financial assets. Which of the following accurately describes changes in the money market the interest rate and the value of the dollar in foreign currency markets? MONEY MARKET     INTEREST RATE     DOLLAR\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}