{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"630498a8-1235-5dec-9ca7-96f39056d4d1","task_key":"default--test--6854","task_revision_id":"1","upstream_id":"6854","short_description":"Suppose the government decides that in order to fight inflation, labor unions…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['imported inflation', 'hyperinflation', 'built-in inflation', 'sectoral inflation', 'asset inflation', 'stagflation', 'deflation', 'wage inflation', 'cost-push inflation', 'demand-pull inflation']\",\"question\":\"Suppose the government decides that in order to fight inflation, labor unions will not be allowed any wage increases until further notice. What type of inflation would this be aimed at?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}