{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"acadc867-3057-53a2-93d7-49ddd7d42ecc","task_key":"default--test--7036","task_revision_id":"1","upstream_id":"7036","short_description":"If the standard of living increases we can conclude that","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['output must have increased proportionally more than population.', 'the cost of living and population must have increased.', 'the cost of living must have decreased.', 'population must have decreased.', 'output and population must have increased.', 'output and cost of living must have increased.', 'output and population must have decreased.', 'population must have increased.', 'population must have increased proportionally more than output.']\",\"question\":\"If the standard of living increases we can conclude that\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}