{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8bc82ee5-71a6-5b4c-8268-c589061849a0","task_key":"default--test--6940","task_revision_id":"1","upstream_id":"6940","short_description":"Suppose that the public wishes to hold 15 percent of its total money balances as…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['$ 100 billion', '$ 50.4 billion', '$ 170 billion', '$ 60 billion', '$ 30 billion', '$ 24 billion', '$ 42 billion', '$ 75 billion', '$ 20.4 billion', '$ 36 billion']\",\"question\":\"Suppose that the public wishes to hold 15 percent of its total money balances as cash, the remainder in the form of bank deposits, and banks hold cash reserves equal to 12 per-cent of the deposits. What total amount of cash will be re-quired, for use both by the banks and the public, for a total money supply of $ 200 billion?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}