{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e31bcd6c-2314-58c7-b116-c201b1e3a7f3","task_key":"default--test--7348","task_revision_id":"1","upstream_id":"7348","short_description":"When both short-run aggregate supply and aggregate demand increase which of the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Both the price level and real GDP fall.', 'The price level rises but the change in real GDP is uncertain.', 'The price level falls but the change in real GDP is uncertain.', 'Real GDP falls but the change in the price level is uncertain.', 'The price level rises but real GDP falls.', 'The price level rises and real GDP remains unchanged.', 'The price level falls but real GDP rises.', 'Both the price level and real GDP remain unchanged.', 'Real GDP rises but the change in the price level is uncertain.', 'Both the price level and real GDP rise.']\",\"question\":\"When both short-run aggregate supply and aggregate demand increase which of the following can be said for certain?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}