{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e49f6d4-6812-5962-98ce-19c95839c3a0","task_key":"default--test--6994","task_revision_id":"1","upstream_id":"6994","short_description":"What gives rise to the difference in the aggregate demand for a nontrading or…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Differences in taxation policies between economies', 'The inclusion or exclusion of foreign direct investment', 'absence of investment', 'Variances in the level of technological advancement', 'Variations in the savings rate of the population', 'absence of the trade components', 'The presence or absence of a central bank', 'Fluctuations in the exchange rate', 'no consumption', 'lack of government purchases']\",\"question\":\"What gives rise to the difference in the aggregate demand for a nontrading or closed economy from a trading or open economy?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}