{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f12d3be2-6963-58f5-9c76-6ba22603e2f2","task_key":"default--test--7627","task_revision_id":"1","upstream_id":"7627","short_description":"According to Keynesian theory a decrease in the money supply would","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['raise interest rates which would encourage saving and therefore increase spending.', 'lower interest rates which would discourage borrowing and therefore reduce spending.', 'lower interest rates which would discourage saving and therefore increase spending.', 'raise interest rates which would discourage borrowing and therefore increase spending.', 'have no impact on interest rates but would increase spending.', 'have no impact on interest rates but would decrease spending.', 'raise interest rates which would discourage borrowing and therefore reduce spending.', 'lower interest rates which would encourage saving and therefore reduce spending.', 'raise interest rates which would encourage borrowing and therefore increase spending.', 'lower interest rates which would encourage borrowing and therefore increase spending.']\",\"question\":\"According to Keynesian theory a decrease in the money supply would\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}