{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18790159-ecfe-575b-83b1-c3a467136f5d","task_key":"default--test--7641","task_revision_id":"1","upstream_id":"7641","short_description":"When nominal GDP is rising we would expect money demand to","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['increase as consumers demand more money for transactions increasing the interest rate.', 'increase as consumers demand more money as a financial asset increasing the interest rate.', 'remain constant as the increase in GDP does not impact the demand for money.', 'decrease as consumers demand less money for transactions decreasing the interest rate.', 'remain constant as consumers demand the same amount of money for transactions, keeping the interest rate steady.', 'decrease as the purchasing power of the dollar is rising, increasing the interest rate.', 'increase as the purchasing power of the dollar is rising, decreasing the interest rate.', 'decrease as consumers demand more money for transactions increasing the interest rate.', 'increase as consumers demand less money as a financial asset, decreasing the interest rate.', 'decrease as the purchasing power of the dollar is falling decreasing the interest rate.']\",\"question\":\"When nominal GDP is rising we would expect money demand to\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}