{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e23422aa-2a94-5bf9-9cad-330208ee8db4","task_key":"default--test--7230","task_revision_id":"1","upstream_id":"7230","short_description":"According to the quantity theory of money increasing the money supply serves to","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['increase long-run output with no significant impact on inflation.', 'have no significant impact on either output or inflation.', 'lower the unemployment rate while also lowering the rate of inflation.', 'increase short-run output but it is the source of long-run inflation.', \\\"increase the nation's long-run capacity to produce.\\\", 'stimulate long-run production and employment with significant short-run inflation.', 'decrease the rate of inflation with no significant impact on output.', 'stimulate short-run production and employment with very little long-run inflation.', 'increase the unemployment rate while also raising the rate of inflation.', 'decrease both short-run output and long-run inflation.']\",\"question\":\"According to the quantity theory of money increasing the money supply serves to\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}