{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce4ba3ce-03f3-597f-9bab-2c3f2f5d0cd5","task_key":"default--test--7554","task_revision_id":"1","upstream_id":"7554","short_description":"A likely cause of falling Treasury bond prices might be","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['an increase in private sector investment.', 'expansionary monetary policy.', 'a strengthening dollar.', 'fiscal policy designed to reduce the budget deficit.', 'contractionary monetary policy.', 'a rise in the stock market.', 'a decrease in the unemployment rate.', 'an increase in consumer spending.', 'a depreciating dollar.', 'a decrease in the inflation rate.']\",\"question\":\"A likely cause of falling Treasury bond prices might be\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}