{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7337a6d7-9f1b-5a47-bf41-1082a7ff5d3e","task_key":"default--test--6956","task_revision_id":"1","upstream_id":"6956","short_description":"In what ways is contractionary fiscal policy in the United States likely to…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Interest rates increase and the dollar depreciates.', 'Interest rates remain the same and the dollar depreciates.', 'Interest rates decrease and the dollar appreciates.', 'Interest rates and the value of the dollar both remain the same.', 'Interest rates increase but the value of the dollar remains the same.', 'Interest rates remain the same and the dollar appreciates.', 'There is no predictable effect on either interest rates or the value of the dollar.', 'Interest rates decrease and the dollar depreciates.', 'Interest rates increase and the dollar appreciates.', 'Interest rates decrease but the value of the dollar remains the same.']\",\"question\":\"In what ways is contractionary fiscal policy in the United States likely to affect domestic interest rates and the international value of the dollar?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}