{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4d2acdc-488d-503f-a47d-c2e08de241ad","task_key":"default--test--6855","task_revision_id":"1","upstream_id":"6855","short_description":"What important aspects of the workings of a capitalist economy are not captured…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Includes all aspects of a capitalist economy', 'Provides detailed explanation of flows', 'Omits description of principal elements, does not explain composition of flows, ignores price determination, cannot illustrate fluctuations in employment, output, and income, omits public sector, and does not highlight the effects of the use of money', 'Includes the effects of governmental operations and policies']\",\"question\":\"What important aspects of the workings of a capitalist economy are not captured in circular flow models?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}