{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9aed333-5ebd-5a35-8f40-b5c377dfc031","task_key":"default--test--7632","task_revision_id":"1","upstream_id":"7632","short_description":"Which of the following creates the trade-offdepicted by the Phillips curve?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A decrease in input costs that corresponds with a decrease in unemployment', 'A decrease in output that corresponds with an increase in the price level', 'An increase in input costs that corresponds with an increase in unemployment', 'A rightward shift in the aggregate demand curve', 'An increase in output that corresponds with a decrease in the price level', 'A decrease in input costs that corresponds with an increase in output', 'An increase in output that corresponds with a decrease in unemployment', 'A leftward shift in the aggregate supply curve', 'A leftward shift in the aggregate demand curve', 'A rightward shift in the aggregate supply curve']\",\"question\":\"Which of the following creates the trade-offdepicted by the Phillips curve?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}