{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4bda2c25-47a7-58dd-8bc2-fed6eead3877","task_key":"default--test--7149","task_revision_id":"1","upstream_id":"7149","short_description":"Suppose the economy is operating beyond full employment. Which of the following…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The government should implement policies to increase aggregate demand.', 'The short-run aggregate supply curve is horizontal.', 'Further increases in aggregate demand will decrease inflation.', 'Further increases in aggregate demand will not lower the unemployment rate but will create inflation.', 'The economy will naturally return to full employment without any intervention.', 'A decrease in aggregate demand will result in a lower price level if prices are sticky.', 'The short-run aggregate supply curve is vertical.', 'The economy is at risk of entering a recession.', 'Further increases in aggregate demand will result in a lower price level.', 'A decrease in aggregate demand will lead to an increase in the price level.']\",\"question\":\"Suppose the economy is operating beyond full employment. Which of the following is true at this point?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}