{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bc76f19-d398-5db4-b40f-87c56199df5d","task_key":"default--test--7536","task_revision_id":"1","upstream_id":"7536","short_description":"Economic stabilization policy usually works with time-lags. Which lags in the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Decision lag and effect lag', 'Recognition lag and implementation lag', 'Implementation lag and evaluation lag', 'Communication lag and stabilization lag', 'Action lag and outside lag', 'Inside time-lag and outside lag', 'Administrative lag and operational lag', 'Policy lag and response lag', 'Recognition lag and decision lag', 'Inside lag and decision lag']\",\"question\":\"Economic stabilization policy usually works with time-lags. Which lags in the effects of economic policy may you discern?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}