{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc5a1cd9-1df4-57cf-b782-c09a709152b0","task_key":"default--test--6894","task_revision_id":"1","upstream_id":"6894","short_description":"In a given period the average price level in the country of Sherwood tripled and…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(G) Stayed the same \\\\xa0\\\\xa0\\\\xa0 Increased', '(B) Increased \\\\xa0\\\\xa0\\\\xa0 Increased', '(D) Increased \\\\xa0\\\\xa0\\\\xa0 Stayed the same', '(J) Increased \\\\xa0\\\\xa0\\\\xa0 Fluctuated', '(F) Stayed the same \\\\xa0\\\\xa0\\\\xa0 Decreased', '(A) Increased \\\\xa0\\\\xa0\\\\xa0 Decreased', '(C) Decreased \\\\xa0\\\\xa0\\\\xa0 Decreased', '(I) Stayed the same \\\\xa0\\\\xa0\\\\xa0 Stayed the same', '(H) Decreased \\\\xa0\\\\xa0\\\\xa0 Stayed the same', '(E) Decreased \\\\xa0\\\\xa0\\\\xa0 Increased']\",\"question\":\"In a given period the average price level in the country of Sherwood tripled and Robin's income increased from $30000 to $60000. What happened to Robin's nominal and real income? Nominal Income     Real Income\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}