{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c129db27-ec3b-590e-a090-dff80cad04b4","task_key":"default--test--6833","task_revision_id":"1","upstream_id":"6833","short_description":"The GDP Deflator differs from the CPI in that the GDP Deflator","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['excludes goods and services not bought by consumers', 'incorporates current year quantities in its calculations', 'only incorporates current year prices in its calculations', 'only incorporates base year quantities in its calculations', 'is not affected by the change in prices over time', 'incorporates both current year prices and base year prices', 'is thought to slightly underestimate the inflation rate', 'uses both current and base year quantities in its calculations', 'is thought to slightly overestimate the inflation rate', 'uses base year quantities in its calculations']\",\"question\":\"The GDP Deflator differs from the CPI in that the GDP Deflator\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}