{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bbd1635d-8f51-5f8b-a70d-5197e6e1edf8","task_key":"default--test--7162","task_revision_id":"1","upstream_id":"7162","short_description":"Having a fractional reserve banking system means that","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"[\\\"the central bank holds a certain fraction of the bank's deposits on reserve at all times\\\", 'a bank cannot lend out all of its deposits', 'the bank must hold a certain percentage of its deposits in reserve at all times', 'the central bank holds a certain fraction of GDP on reserve at all times', 'the central bank reserves the right to set the interest rate on any loan at any time.', 'a bank can lend out all of its deposits but must maintain a certain level of reserves', 'all loans must be backed by an equivalent amount of reserves', 'no single loan can be larger than 20 percent of the bank’s holdings', 'a bank can only lend out a fixed percentage of its deposits', 'the central bank reserves the right to audit any bank at any time']\",\"question\":\"Having a fractional reserve banking system means that\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}