{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3f5ee37-9b93-5efa-9330-90bae8b53a51","task_key":"default--test--7624","task_revision_id":"1","upstream_id":"7624","short_description":"When a nation is operating at the natural rate of unemployment","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the nation is in a state of deflation.', 'the nation is experiencing an economic boom.', 'there is no structural unemployment.', 'there is no cyclical unemployment.', 'the nation is experiencing a recession.', 'the inflation rate is zero.', 'the nation is at its highest rate of employment.', \\\"the nation's GDP is at its peak.\\\", 'the nation is experiencing hyperinflation.', 'there is no frictional unemployment.']\",\"question\":\"When a nation is operating at the natural rate of unemployment\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}