{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d71c81d6-1bfb-5265-aff9-36c439bbddc3","task_key":"default--test--7605","task_revision_id":"1","upstream_id":"7605","short_description":"Given the following national income accounting data, compute GNP, NNP and NI…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['GNP = 350.7, NNP = 338.5, NI = 363.9', 'GNP = 325.4, NNP = 351.6, NI = 363.9', 'GNP = 363.9, NNP = 351.6, NI = 338.5', 'GNP = 338.5, NNP = 363.9, NI = 351.6', 'GNP = 338.5, NNP = 351.6, NI = 363.9', 'GNP = 351.6, NNP = 363.9, NI = 338.5', 'GNP = 351.6, NNP = 338.5, NI = 363.9', 'GNP = 376.1, NNP = 363.8, NI = 350.7', 'GNP = 363.9, NNP = 338.5, NI = 325.4', 'GNP = 363.9, NNP = 338.5, NI = 351.6']\",\"question\":\"Given the following national income accounting data, compute GNP, NNP and NI (all figures are in billions). - Compensation of employees195.6 - U.S. exports of goods and services14.5 - Capital consumption allowance12.3 - Government purchases of goods and services60.6 - Indirect business taxes13.1 - Net private domestic investment63.4 - Transfer payments14.2 - U.S. imports of goods and services17.8 - Personal taxes43.0 - Personal consumption expenditures230.9\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}