{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44395f56-49e1-5d96-87da-0b0291ca1ca0","task_key":"default--test--7161","task_revision_id":"1","upstream_id":"7161","short_description":"The potential amount of money created after the Fed increases bank reserves will…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the public prefers to hold less cash.', 'the Fed decreases the money supply.', 'depository institutions decide to hold less excess reserves.', 'the velocity of money increases.', 'depository institutions decide to hold more excess reserves.']\",\"question\":\"The potential amount of money created after the Fed increases bank reserves will be diminished if\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}