{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"609c4cee-9fb4-54ce-aeb3-a85be929aac2","task_key":"default--test--7281","task_revision_id":"1","upstream_id":"7281","short_description":"When GNP grows by 3.5%, the labor productivity by 1.6% and the labor force by…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Unemployment is decreasing', 'Unemployment decreases at a rate directly proportional to the increase in labor productivity', 'Unemployment is cyclical and independent of GNP, labor productivity, and labor force growth rates', 'Unemployment increases initially but stabilizes over time', 'Unemployment decreases initially but increases over time', 'Unemployment is increasing', 'The rate of unemployment cannot be determined with the given data', 'Unemployment is fluctuating', 'Unemployment remains the same', 'Unemployment is inversely proportional to GNP growth']\",\"question\":\"When GNP grows by 3.5%, the labor productivity by 1.6% and the labor force by 2.2% per year, is unemployment increasing or decreasing?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}