{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"773dc64b-f698-5ef9-a0e7-35160834c319","task_key":"default--test--7277","task_revision_id":"1","upstream_id":"7277","short_description":"What is the consequenceof permanent inflation for the What is the consequence…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Interest rates decrease to stimulate economic growth', 'Interest rates become discounted by the anticipated rate of inflation', 'Inflation has no relation to interest rates', 'Interest rates fluctuate randomly regardless of inflation', 'Central banks have less control over interest rates', 'Interest rates increase dramatically', 'Interest rates consistently drop below the rate of inflation', 'The gap between nominal and real interest rates narrows significantly', 'Interest rates are periodically set to zero to counteract inflation', 'Interest rates are unaffected by inflation']\",\"question\":\"What is the consequenceof permanent inflation for the What is the consequence interest rates?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}