{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1452e508-8f0d-5540-83a0-358e0a7affc2","task_key":"default--test--7404","task_revision_id":"1","upstream_id":"7404","short_description":"GNP attempts to measure the annual production of the econ-omy. Non-productive…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Transactions exclusively involving agricultural products', 'Exports and imports of goods and services', 'Consumer spending on durable goods', 'Government spending on infrastructure projects', 'Investments in education and healthcare', 'public and private transfer payments', 'Sales of new manufactured goods and services', 'transactions involving current production', 'wage payments and stock market transactions', 'purely financial transactions and second-hand sales']\",\"question\":\"GNP attempts to measure the annual production of the econ-omy. Non-productive transactions should not be included in its computation. What are 'non-productive transactions'?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}