{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ebd616d3-0156-5bb4-8434-df766f30912e","task_key":"default--test--6194","task_revision_id":"1","upstream_id":"6194","short_description":"On a global scale, trends are suggesting that with rising incomes and…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Meat and Dairy products', 'Grains and pulses', 'Seafood and poultry', 'Oils and fats', 'Fruits and vegetables', 'Nuts and seeds', 'Beans and lentils', 'Sugars and sweeteners', 'Herbs and spices', 'Roots and Tubers']\",\"question\":\"On a global scale, trends are suggesting that with rising incomes and urbanization we are seeing a rise in which food commodities?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}