{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d7d11fb5-7b8a-5b65-911b-277768db3630","task_key":"default--test--7255","task_revision_id":"1","upstream_id":"7255","short_description":"What happens to the purchasing power of money when there is inflation? What does…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Purchasing power increases and more goods can be purchased with the same amount of money.', 'Purchasing power remains the same.', 'Purchasing power decreases and less goods can be purchased with the same amount of money.', 'Purchasing power fluctuates unpredictably.']\",\"question\":\"What happens to the purchasing power of money when there is inflation? What does it mean?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}