{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f04b5498-eee5-5ab9-8c54-778a62cfdccf","task_key":"default--test--7618","task_revision_id":"1","upstream_id":"7618","short_description":"Why is it that the two decades prior to World War Ihave beendubbed \"the golden…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The demand for farm products, farm prices, and farm incomes all rose.', 'Urbanization led to a decrease in the farming workforce.', 'The federal government imposed high taxes on farm income.', 'The government introduced significant restrictions on crop exports.', 'Land grants for farming were abolished.', 'The industrial revolution started.', 'The introduction of widespread commercial fertilizer use improved crop yields.', 'The Great Depression ended.', 'There was a major decline in agricultural technology.', 'The population in the country decreased.']\",\"question\":\"Why is it that the two decades prior to World War Ihave beendubbed \\\"the golden age of American agriculture\\\"?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}