{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc470d9b-4a7e-5432-911b-3fe24df786f0","task_key":"default--test--6942","task_revision_id":"1","upstream_id":"6942","short_description":"In the extreme case, $ 1,000 of new reserves put into a bank will ultimately…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A decrease in the money supply by the central bank', 'Withdrawals of reserves from the banking system and not adhering to the reserve ratio', 'Low bank deposits', 'Excessive regulation of the banking industry', 'Increase in government borrowing crowding out private lending']\",\"question\":\"In the extreme case, $ 1,000 of new reserves put into a bank will ultimately result in an increase of $ 5,000 of bank de-posits, assuming that the reserve ratio is 20%. But in reality this is rarely the case. What stops the extreme case from being realized?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}