{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8235c39-e30a-5e99-9da2-05b0a7fcd92c","task_key":"default--test--7030","task_revision_id":"1","upstream_id":"7030","short_description":"Congress has embarked on another round of expansionary fiscal policy to boost…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Buy government securities risking a recessionary gap', 'Increase the discount rate risking deflation', 'Sell less government securities risking deflation', 'Increase the reserve ratio risking the devaluation of the dollar', 'Lower the reserve ratio risking a devaluation of the dollar', 'Decrease the reserve ratio risking inflation', 'Sell government securities risking inflation', 'Lower the discount rate risking inflation', 'Buy more government securities risking hyperinflation', 'Increase government spending risking a budget deficit']\",\"question\":\"Congress has embarked on another round of expansionary fiscal policy to boost employment and get reelected. As chair of the central bank how would you reduce the \\\"crowding-out\\\" effect and what macroeconomic problem might your policy exacerbate?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}