{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"670c8737-d72f-54c3-acac-5c22096d69ea","task_key":"default--test--6931","task_revision_id":"1","upstream_id":"6931","short_description":"Define Gross National Product (GNP).","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"[\\\"The total income earned by a nation's residents in a year\\\", 'The total amount of money in circulation in an economy in a year', 'The total market value of all final goods and services produced in the economy in one year', 'The market value of all goods and services produced abroad by the residents of a nation in a year', 'The total cost of all goods and services purchased in a year', \\\"The total savings rate of a nation's residents plus the value of imports minus the value of exports in a year\\\", 'The aggregate of all wages paid to employees, plus profits of businesses and taxes, minus any subsidies', \\\"The sum of all financial transactions within a country's borders in a year\\\", 'The total value of all consumer spending, government spending, investments, and net exports in a year', \\\"The total value of all goods and services produced by a nation's residents, regardless of the location\\\"]\",\"question\":\"Define Gross National Product (GNP).\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}